2024-09-16T01:44:16.982 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T01:44:17.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T01:44:17.092 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/direct_io}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906496' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP6ePn+zz2PLxeOpkNz3UMMV5QZsdoHpSqlgQ1MrNjnbZ2w7paBL+txtSOsq7HKXAyAcxx9nVyE2SBgWQvJaJts= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDFbVwG3yUMipQonY5Tgz+/zxYO5gEQfZY7HiDn22lSDUDeoOafA3p/tw8Tam0iW075ju6MzImm36cnqYp8Ug/Y= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC3jldJJ4fYowVnlRLvMu17H2/oQ1KRk23ODgFovY7kZ686Yu609zv/tzdbS+2CXWDfHvuYBGijV0X5ymXz2P58= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T01:44:17.093 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-16T01:44:17.093 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-16T01:44:17.093 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T01:44:17.095 INFO:teuthology.task.internal:Checking packages... 2024-09-16T01:44:17.119 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-16T01:44:17.119 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T01:44:17.119 INFO:teuthology.packaging:ref: None 2024-09-16T01:44:17.119 INFO:teuthology.packaging:tag: None 2024-09-16T01:44:17.119 INFO:teuthology.packaging:branch: squid 2024-09-16T01:44:17.119 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:44:17.119 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-16T01:44:17.336 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-16T01:44:17.337 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T01:44:17.344 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T01:44:17.344 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T01:44:17.359 INFO:teuthology.task.internal:Saving configuration 2024-09-16T01:44:17.379 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T01:44:17.386 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T01:44:17.407 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 01:38:54.687101', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP6ePn+zz2PLxeOpkNz3UMMV5QZsdoHpSqlgQ1MrNjnbZ2w7paBL+txtSOsq7HKXAyAcxx9nVyE2SBgWQvJaJts='} 2024-09-16T01:44:17.429 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 01:38:54.690629', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDFbVwG3yUMipQonY5Tgz+/zxYO5gEQfZY7HiDn22lSDUDeoOafA3p/tw8Tam0iW075ju6MzImm36cnqYp8Ug/Y='} 2024-09-16T01:44:17.447 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 01:38:54.688890', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC3jldJJ4fYowVnlRLvMu17H2/oQ1KRk23ODgFovY7kZ686Yu609zv/tzdbS+2CXWDfHvuYBGijV0X5ymXz2P58='} 2024-09-16T01:44:17.447 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T01:44:17.454 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-16T01:44:17.454 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-16T01:44:17.455 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-16T01:44:17.455 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T01:44:17.532 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f23a301f490>, signals=[15]) 2024-09-16T01:44:17.532 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T01:44:17.540 INFO:teuthology.task.internal:Opening connections... 2024-09-16T01:44:17.540 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-09-16T01:44:17.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:44:17.627 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T01:44:17.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:44:17.708 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-09-16T01:44:17.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:44:17.789 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T01:44:17.796 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-09-16T01:44:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-09-16T01:44:17.816 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-09-16T01:44:17.873 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-09-16T01:44:17.874 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-09-16T01:44:17.874 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-09-16T01:44:17.874 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-09-16T01:44:17.874 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-09-16T01:44:17.874 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-09-16T01:44:17.874 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T01:44:17.874 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-09-16T01:44:17.875 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-09-16T01:44:17.875 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T01:44:17.875 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-09-16T01:44:17.875 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T01:44:17.875 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T01:44:17.875 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T01:44:17.876 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-09-16T01:44:17.894 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-09-16T01:44:17.912 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-09-16T01:44:17.913 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-09-16T01:44:17.971 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-09-16T01:44:17.971 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-09-16T01:44:17.971 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-09-16T01:44:17.971 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-09-16T01:44:17.971 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-09-16T01:44:17.971 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-09-16T01:44:17.971 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T01:44:17.971 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-09-16T01:44:17.972 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-09-16T01:44:17.972 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T01:44:17.972 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-09-16T01:44:17.972 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T01:44:17.972 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T01:44:17.972 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T01:44:17.973 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-09-16T01:44:17.994 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-09-16T01:44:18.013 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-09-16T01:44:18.013 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T01:44:18.070 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2024-09-16T01:44:18.071 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T01:44:18.071 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T01:44:18.071 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T01:44:18.071 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-09-16T01:44:18.094 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T01:44:18.103 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T01:44:18.109 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T01:44:18.110 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-09-16T01:44:18.112 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-09-16T01:44:18.115 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-09-16T01:44:18.132 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T01:44:18.139 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T01:44:18.140 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-09-16T01:44:18.170 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-09-16T01:44:18.173 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-09-16T01:44:18.377 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T01:44:18.442 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T01:44:18.457 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-16T01:44:18.457 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-16T01:44:18.457 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-16T01:44:18.458 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T01:44:18.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:44:18.476 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-16T01:44:18.641 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T01:44:18.642 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-16T01:44:18.642 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-16T01:44:18.660 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-503.el9.x86_64 2024-09-16T01:44:18.660 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T01:44:18.661 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-16T01:44:18.661 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum upgrade -y linux-firmware 2024-09-16T01:44:23.432 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.2 MB 00:02 2024-09-16T01:44:25.108 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 37 MB/s | 20 MB 00:00 2024-09-16T01:44:30.392 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-16T01:44:32.035 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-16T01:44:32.872 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-16T01:44:38.378 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T01:44:40.469 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-16T01:44:40.470 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-16T01:44:40.470 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-16T01:44:40.582 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-16T01:44:40.583 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T01:44:40.732 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-16T01:44:40.733 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-16T01:44:40.735 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi103.front.sepia.ceph.com to complete... 2024-09-16T01:44:47.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16 01:44:47 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-16T01:44:47.036 INFO:teuthology.orchestra.run.smithi103.stderr:FINISHED --2024-09-16 01:44:47-- 2024-09-16T01:44:47.036 INFO:teuthology.orchestra.run.smithi103.stderr:Total wall clock time: 6.3s 2024-09-16T01:44:47.036 INFO:teuthology.orchestra.run.smithi103.stderr:Downloaded: 1 files, 343M in 6.0s (57.3 MB/s) 2024-09-16T01:44:47.039 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-16T01:44:47.040 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-16T01:44:49.150 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying... ######################################## 2024-09-16T01:44:49.300 INFO:teuthology.orchestra.run.smithi103.stdout:Preparing... ######################################## 2024-09-16T01:44:49.411 INFO:teuthology.orchestra.run.smithi103.stdout:Updating / installing... 2024-09-16T01:44:57.529 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-16T01:45:38.003 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi103.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-16T01:45:38.004 DEBUG:teuthology.orchestra.run.smithi103:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-16T01:45:38.096 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:45:38.096 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:45:38.096 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:45:38.096 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:45:38.096 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-16T01:45:38.096 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-16T01:45:38.097 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-16T01:45:38.098 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-16T01:45:38.099 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-16T01:45:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-16T01:45:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-16T01:45:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-16T01:45:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-16T01:45:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-16T01:45:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-16T01:45:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-16T01:45:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-16T01:45:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-16T01:45:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-16T01:45:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-16T01:45:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-16T01:45:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-16T01:45:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-16T01:45:38.106 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-16T01:45:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-16T01:45:38.108 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-16T01:45:38.109 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-16T01:45:38.110 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-16T01:45:38.111 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-16T01:45:38.112 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-16T01:45:38.113 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-16T01:45:38.114 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-16T01:45:38.115 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-16T01:45:38.116 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-16T01:45:38.117 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-16T01:45:38.118 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-16T01:45:38.119 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-16T01:45:38.120 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-16T01:45:38.121 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-16T01:45:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-16T01:45:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-16T01:45:38.124 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-16T01:45:38.125 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-16T01:45:38.126 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-16T01:45:38.127 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-16T01:45:38.128 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-16T01:45:38.129 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-16T01:45:38.130 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-16T01:45:38.131 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-16T01:45:38.132 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-16T01:45:38.133 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-16T01:45:38.134 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-16T01:45:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-16T01:45:38.136 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-16T01:45:38.137 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-16T01:45:38.138 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-16T01:45:38.139 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-16T01:45:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-16T01:45:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-16T01:45:38.142 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-16T01:45:38.143 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-16T01:45:38.144 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-16T01:45:38.145 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-16T01:45:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-16T01:45:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-16T01:45:38.148 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-16T01:45:38.149 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-16T01:45:38.150 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-16T01:45:38.151 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-16T01:45:38.152 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-16T01:45:38.153 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-16T01:45:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-16T01:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-16T01:45:38.156 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-16T01:45:38.157 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-16T01:45:38.158 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-16T01:45:38.159 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-16T01:45:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-16T01:45:38.161 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-16T01:45:38.162 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-16T01:45:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-16T01:45:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-16T01:45:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-16T01:45:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-16T01:45:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-16T01:45:38.168 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-16T01:45:38.169 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-16T01:45:38.170 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-16T01:45:38.171 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-16T01:45:38.172 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-16T01:45:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-16T01:45:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-16T01:45:38.175 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-16T01:45:38.176 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-16T01:45:38.177 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-16T01:45:38.178 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-16T01:45:38.179 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-16T01:45:38.180 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-16T01:45:38.181 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-16T01:45:38.182 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-16T01:45:38.183 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-16T01:45:38.184 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-16T01:45:38.185 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-16T01:45:38.186 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-16T01:45:38.187 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-16T01:45:38.188 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-16T01:45:38.189 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-16T01:45:38.190 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-16T01:45:38.191 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-16T01:45:38.192 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-16T01:45:38.193 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-16T01:45:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-16T01:45:38.195 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-16T01:45:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-16T01:45:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-16T01:45:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-16T01:45:38.199 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-16T01:45:38.200 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-16T01:45:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-16T01:45:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-16T01:45:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-16T01:45:38.204 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-16T01:45:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-16T01:45:38.206 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-16T01:45:38.207 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-16T01:45:38.208 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-16T01:45:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-16T01:45:38.210 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-16T01:45:38.211 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-16T01:45:38.212 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-16T01:45:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-16T01:45:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-16T01:45:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-16T01:45:38.216 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-16T01:45:38.217 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-16T01:45:38.218 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-16T01:45:38.219 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-16T01:45:38.220 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-16T01:45:38.221 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-16T01:45:38.222 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-16T01:45:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-16T01:45:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-16T01:45:38.225 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-16T01:45:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-16T01:45:38.227 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-16T01:45:38.228 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-16T01:45:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-16T01:45:38.230 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-16T01:45:38.231 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-16T01:45:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-16T01:45:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-16T01:45:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-16T01:45:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-16T01:45:38.236 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-16T01:45:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-16T01:45:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-16T01:45:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-16T01:45:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-16T01:45:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-16T01:45:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-16T01:45:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-16T01:45:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-16T01:45:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T01:45:38.242 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:45:38.242 DEBUG:teuthology.orchestra.run.smithi103:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-16T01:45:38.266 INFO:teuthology.orchestra.run.smithi103.stdout:postinstall scriptlet (using /bin/sh): 2024-09-16T01:45:38.266 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T01:45:38.266 INFO:teuthology.orchestra.run.smithi103.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T01:45:38.266 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout:for file in vmlinuz System.map config; do 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: fi 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout:done 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:45:38.267 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-09-16T01:45:38.268 INFO:teuthology.orchestra.run.smithi103.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-16T01:45:38.268 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T01:45:38.268 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:45:38.268 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-09-16T01:45:38.269 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -qi grub2-tools 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Name : grub2-tools 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Epoch : 1 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Version : 2.06 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Release : 82.el9 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Architecture: x86_64 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Group : Unspecified 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Size : 8274670 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:License : GPLv3+ 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T01:45:38.343 INFO:teuthology.orchestra.run.smithi103.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:Packager : builder@centos.org 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:Vendor : CentOS 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:Summary : Support tools for GRUB. 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:Description : 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:hardware devices. 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:45:38.344 INFO:teuthology.orchestra.run.smithi103.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T01:45:38.345 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T01:45:38.346 INFO:teuthology.task.kernel:Updating grub on smithi103 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:45:38.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T01:45:38.895 INFO:teuthology.orchestra.run.smithi103.stderr:Generating grub configuration file ... 2024-09-16T01:45:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T01:45:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:done 2024-09-16T01:45:40.310 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-09-16T01:45:40.325 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.hpZVLYGoyy 2024-09-16T01:45:40.326 DEBUG:teuthology.orchestra.run.smithi103:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hpZVLYGoyy 2024-09-16T01:45:40.393 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.hpZVLYGoyy 2024-09-16T01:45:40.524 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.hpZVLYGoyy is 6KB 2024-09-16T01:45:40.572 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.hpZVLYGoyy 2024-09-16T01:45:40.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T01:45:40.650 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T01:45:40.650 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T01:45:40.650 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T01:45:40.650 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-16T01:45:40.652 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:45:40.803 DEBUG:teuthology.orchestra.run.smithi103:> sudo shutdown -r now 2024-09-16T01:46:10.805 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-16T01:46:10.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-16T01:46:10.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:46:45.437 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-16T01:46:54.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-16T01:46:54.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:47:54.442 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T01:48:06.444 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-16T01:48:06.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:48:06.736 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-16T01:48:07.224 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-16T01:48:07.224 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-16T01:48:07.224 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-16T01:48:07.259 INFO:teuthology.orchestra.run.smithi103.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:48:07.259 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T01:48:07.259 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T01:48:07.259 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T01:48:08.260 DEBUG:teuthology.parallel:result is None 2024-09-16T01:48:08.260 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T01:48:08.267 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T01:48:08.268 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T01:48:08.272 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T01:48:08.275 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T01:48:08.306 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T01:48:08.312 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T01:48:08.378 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T01:48:08.378 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T01:48:08.382 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T01:48:08.384 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T01:48:08.435 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T01:48:08.442 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T01:48:08.443 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T01:48:08.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:48:08.473 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T01:48:08.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:48:08.491 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T01:48:08.495 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T01:48:08.517 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T01:48:08.556 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:48:08.562 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:48:08.573 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:48:08.588 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:48:08.590 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:48:08.618 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:48:08.621 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T01:48:08.628 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T01:48:08.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T01:48:08.632 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T01:48:08.634 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T01:48:08.684 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T01:48:08.694 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T01:48:08.695 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T01:48:08.699 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T01:48:08.703 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T01:48:08.729 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:48:08.836 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:48:08.918 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:48:08.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:48:09.051 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:48:09.051 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T01:48:09.106 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:48:09.150 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:48:09.206 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:48:09.288 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:48:09.333 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:48:09.333 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T01:48:09.400 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:48:09.445 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:48:09.492 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:48:09.576 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:48:09.620 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:48:09.620 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T01:48:09.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-09-16T01:48:09.692 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-09-16T01:48:09.695 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-09-16T01:48:09.733 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:48:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:48:09.767 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:48:10.080 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T01:48:10.089 INFO:teuthology.task.internal:Starting timer... 2024-09-16T01:48:10.089 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T01:48:10.101 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T01:48:10.163 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-16T01:48:10.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-09-16T01:48:10.252 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-09-16T01:48:10.256 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-09-16T01:48:10.315 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-09-16T01:48:10.317 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2024-09-16T01:48:10.379 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2024-09-16T01:48:10.381 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T01:48:10.381 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-09-16T01:48:10.423 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-09-16T01:48:10.423 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-09-16T01:48:10.459 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-09-16T01:48:10.459 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2024-09-16T01:48:10.480 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2024-09-16T01:48:10.480 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive'} 2024-09-16T01:48:10.480 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:48:10.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:48:10.538 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:48:10.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:48:10.569 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:48:10.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:48:10.598 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T01:48:10.598 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-09-16T01:48:10.658 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-09-16T01:48:10.699 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2024-09-16T01:48:10.733 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T01:48:10.747 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T01:48:10.757 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T01:48:10.759 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi103.front.sepia.ceph.com,smithi136.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T01:52:47.859 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2024-09-16T01:52:47.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-16T01:52:47.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:52:47.954 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-16T01:52:48.063 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-16T01:52:48.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-16T01:52:48.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:52:48.145 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-09-16T01:52:48.233 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-16T01:52:48.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-16T01:52:48.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:52:48.325 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-09-16T01:52:48.400 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-16T01:52:48.401 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T01:52:48.411 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T01:52:48.411 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T01:52:48.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:52:48.416 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T01:52:48.416 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:52:48.419 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T01:52:48.419 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:52:48.451 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T01:52:48.466 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T01:52:48.471 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T01:52:48.473 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T01:52:48.485 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T01:52:48.492 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-09-16T01:52:48.503 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T01:52:48.504 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-09-16T01:52:48.512 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2024-09-16T01:52:48.518 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T01:52:48.521 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2024-09-16T01:52:48.533 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T01:52:48.535 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T01:52:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T01:52:48.555 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-09-16T01:52:48.580 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-09-16T01:52:48.586 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:52:48.589 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:52:48.590 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-09-16T01:52:48.590 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.590 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.590 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.590 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.605 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:52:48.607 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T01:52:48.608 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:52:48.608 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-09-16T01:52:48.609 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.609 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.609 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.609 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.637 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T01:52:48.732 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:52:48.738 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:52:48.738 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-16T01:52:48.738 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.738 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.738 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.738 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:52:48.739 INFO:teuthology.run_tasks:Running task install... 2024-09-16T01:52:48.747 DEBUG:teuthology.task.install:project ceph 2024-09-16T01:52:48.747 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-16T01:52:48.747 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T01:52:48.747 INFO:teuthology.task.install:Using flavor: default 2024-09-16T01:52:48.753 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T01:52:48.753 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T01:52:48.753 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T01:52:48.754 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:52:48.756 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T01:52:48.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:52:48.757 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T01:52:48.757 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:52:48.904 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T01:52:48.905 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T01:52:48.908 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T01:52:48.908 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T01:52:48.911 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T01:52:48.911 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T01:52:49.016 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T01:52:49.016 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:52:49.016 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T01:52:49.023 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T01:52:49.023 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:52:49.024 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T01:52:49.028 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T01:52:49.028 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:52:49.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T01:52:49.042 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T01:52:49.043 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T01:52:49.049 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T01:52:49.050 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T01:52:49.084 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T01:52:49.085 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T01:52:49.109 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T01:52:49.114 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T01:52:49.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T01:52:49.184 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T01:52:49.189 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T01:52:49.248 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2024-09-16T01:52:49.250 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2024-09-16T01:52:49.253 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-09-16T01:52:49.254 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-09-16T01:52:49.261 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T01:52:49.316 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-09-16T01:52:49.319 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-09-16T01:52:49.478 INFO:teuthology.orchestra.run.smithi136.stdout:45 files removed 2024-09-16T01:52:49.485 INFO:teuthology.orchestra.run.smithi172.stdout:45 files removed 2024-09-16T01:52:49.505 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T01:52:49.513 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T01:52:49.591 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-09-16T01:52:49.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T01:52:50.043 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 320 kB/s | 83 kB 00:00 2024-09-16T01:52:50.061 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 313 kB/s | 83 kB 00:00 2024-09-16T01:52:50.303 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 294 kB/s | 83 kB 00:00 2024-09-16T01:52:50.327 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-16T01:52:50.346 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-16T01:52:50.594 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-16T01:52:50.595 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-16T01:52:50.613 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-16T01:52:50.863 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-16T01:52:51.121 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-16T01:52:51.649 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-16T01:52:51.669 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.2 MB 00:01 2024-09-16T01:52:54.346 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 13 MB/s | 20 MB 00:01 2024-09-16T01:52:54.422 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 9.2 MB/s | 20 MB 00:02 2024-09-16T01:52:56.989 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 20 MB 00:04 2024-09-16T01:52:59.304 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 6.5 MB 00:01 2024-09-16T01:53:00.573 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.5 MB 00:02 2024-09-16T01:53:00.972 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-16T01:53:01.934 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-16T01:53:02.098 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-16T01:53:02.665 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-16T01:53:03.319 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-16T01:53:04.324 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 19 kB 00:00 2024-09-16T01:53:05.370 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-16T01:53:07.431 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-16T01:53:08.769 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T01:53:09.803 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:53:09.803 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:53:09.817 INFO:teuthology.orchestra.run.smithi136.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T01:53:09.818 INFO:teuthology.orchestra.run.smithi136.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T01:53:09.819 INFO:teuthology.orchestra.run.smithi136.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T01:53:09.820 INFO:teuthology.orchestra.run.smithi136.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T01:53:09.821 INFO:teuthology.orchestra.run.smithi136.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T01:53:09.824 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T01:53:09.825 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T01:53:09.888 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================== 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================== 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T01:53:09.913 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T01:53:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T01:53:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T01:53:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T01:53:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T01:53:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T01:53:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T01:53:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T01:53:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T01:53:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T01:53:09.914 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T01:53:09.915 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T01:53:09.916 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T01:53:09.917 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T01:53:09.918 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T01:53:09.919 INFO:teuthology.orchestra.run.smithi136.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T01:53:09.920 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T01:53:09.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T01:53:09.922 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T01:53:09.923 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T01:53:09.924 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-09-16T01:53:09.925 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T01:53:09.926 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:53:09.926 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-09-16T01:53:09.926 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================== 2024-09-16T01:53:09.926 INFO:teuthology.orchestra.run.smithi136.stdout:Install 148 Packages 2024-09-16T01:53:09.926 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 8 Packages 2024-09-16T01:53:09.926 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:53:09.926 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 226 M 2024-09-16T01:53:09.926 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-09-16T01:53:10.358 INFO:teuthology.orchestra.run.smithi136.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 48 kB/s | 6.5 kB 00:00 2024-09-16T01:53:10.566 INFO:teuthology.orchestra.run.smithi136.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 4.2 MB/s | 896 kB 00:00 2024-09-16T01:53:10.633 INFO:teuthology.orchestra.run.smithi136.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 2.1 MB/s | 145 kB 00:00 2024-09-16T01:53:10.725 INFO:teuthology.orchestra.run.smithi136.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-16T01:53:10.842 INFO:teuthology.orchestra.run.smithi136.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-16T01:53:10.884 INFO:teuthology.orchestra.run.smithi136.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-16T01:53:11.060 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:53:11.061 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:53:11.073 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T01:53:11.074 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T01:53:11.074 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T01:53:11.075 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T01:53:11.077 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T01:53:11.080 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T01:53:11.080 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T01:53:11.142 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-16T01:53:11.167 INFO:teuthology.orchestra.run.smithi172.stdout:====================================================================================== 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout:====================================================================================== 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T01:53:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T01:53:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T01:53:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T01:53:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T01:53:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T01:53:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T01:53:11.172 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-16T01:53:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T01:53:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T01:53:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T01:53:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T01:53:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T01:53:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T01:53:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T01:53:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T01:53:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T01:53:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T01:53:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T01:53:11.174 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T01:53:11.174 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T01:53:11.174 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T01:53:11.174 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T01:53:11.174 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T01:53:11.174 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T01:53:11.175 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T01:53:11.175 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T01:53:11.175 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T01:53:11.175 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T01:53:11.175 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T01:53:11.175 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T01:53:11.175 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T01:53:11.176 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T01:53:11.176 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T01:53:11.176 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T01:53:11.176 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T01:53:11.176 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T01:53:11.176 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T01:53:11.176 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T01:53:11.177 INFO:teuthology.orchestra.run.smithi172.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T01:53:11.177 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T01:53:11.177 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T01:53:11.177 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T01:53:11.177 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T01:53:11.177 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T01:53:11.177 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T01:53:11.178 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T01:53:11.178 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T01:53:11.178 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T01:53:11.178 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T01:53:11.178 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T01:53:11.178 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T01:53:11.179 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T01:53:11.179 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T01:53:11.179 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T01:53:11.179 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T01:53:11.179 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T01:53:11.179 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T01:53:11.180 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T01:53:11.180 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T01:53:11.180 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T01:53:11.180 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T01:53:11.180 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T01:53:11.180 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T01:53:11.181 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T01:53:11.181 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T01:53:11.181 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T01:53:11.181 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T01:53:11.181 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T01:53:11.181 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T01:53:11.181 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T01:53:11.181 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T01:53:11.181 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T01:53:11.181 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T01:53:11.182 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T01:53:11.183 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T01:53:11.183 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T01:53:11.183 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T01:53:11.183 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T01:53:11.183 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T01:53:11.183 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T01:53:11.183 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T01:53:11.183 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T01:53:11.184 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T01:53:11.185 INFO:teuthology.orchestra.run.smithi172.stdout:Installing weak dependencies: 2024-09-16T01:53:11.186 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T01:53:11.186 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:53:11.186 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-16T01:53:11.186 INFO:teuthology.orchestra.run.smithi172.stdout:====================================================================================== 2024-09-16T01:53:11.186 INFO:teuthology.orchestra.run.smithi172.stdout:Install 148 Packages 2024-09-16T01:53:11.186 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 8 Packages 2024-09-16T01:53:11.186 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:53:11.187 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 226 M 2024-09-16T01:53:11.187 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-16T01:53:11.229 INFO:teuthology.orchestra.run.smithi136.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-16T01:53:11.526 INFO:teuthology.orchestra.run.smithi136.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 16 MB/s | 21 MB 00:01 2024-09-16T01:53:11.568 INFO:teuthology.orchestra.run.smithi136.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 610 kB/s | 25 kB 00:00 2024-09-16T01:53:11.685 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-09-16T01:53:11.809 INFO:teuthology.orchestra.run.smithi172.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 29 kB/s | 6.5 kB 00:00 2024-09-16T01:53:11.877 INFO:teuthology.orchestra.run.smithi136.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 17 MB/s | 17 MB 00:00 2024-09-16T01:53:12.010 INFO:teuthology.orchestra.run.smithi136.stdout:(11/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 14 MB/s | 11 MB 00:00 2024-09-16T01:53:12.035 INFO:teuthology.orchestra.run.smithi136.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.4 MB/s | 228 kB 00:00 2024-09-16T01:53:12.061 INFO:teuthology.orchestra.run.smithi136.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 539 kB/s | 27 kB 00:00 2024-09-16T01:53:12.086 INFO:teuthology.orchestra.run.smithi136.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 628 kB/s | 32 kB 00:00 2024-09-16T01:53:12.128 INFO:teuthology.orchestra.run.smithi136.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 751 kB 00:00 2024-09-16T01:53:12.153 INFO:teuthology.orchestra.run.smithi136.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.4 MB/s | 163 kB 00:00 2024-09-16T01:53:12.159 INFO:teuthology.orchestra.run.smithi172.stdout:(2/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 9.4 MB/s | 5.4 MB 00:00 2024-09-16T01:53:12.179 INFO:teuthology.orchestra.run.smithi136.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-16T01:53:12.229 INFO:teuthology.orchestra.run.smithi136.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 6.6 MB/s | 506 kB 00:00 2024-09-16T01:53:12.234 INFO:teuthology.orchestra.run.smithi172.stdout:(3/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 2.1 MB/s | 896 kB 00:00 2024-09-16T01:53:12.260 INFO:teuthology.orchestra.run.smithi172.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 1.4 MB/s | 145 kB 00:00 2024-09-16T01:53:12.313 INFO:teuthology.orchestra.run.smithi136.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 541 kB/s | 45 kB 00:00 2024-09-16T01:53:12.510 INFO:teuthology.orchestra.run.smithi172.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 6.1 MB/s | 1.5 MB 00:00 2024-09-16T01:53:12.571 INFO:teuthology.orchestra.run.smithi136.stdout:(20/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-16T01:53:12.638 INFO:teuthology.orchestra.run.smithi136.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.4 MB/s | 161 kB 00:00 2024-09-16T01:53:12.722 INFO:teuthology.orchestra.run.smithi136.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.8 MB/s | 320 kB 00:00 2024-09-16T01:53:12.789 INFO:teuthology.orchestra.run.smithi136.stdout:(23/156): python3-ceph-common-19.1.1-326.g8921c 286 kB/s | 136 kB 00:00 2024-09-16T01:53:12.814 INFO:teuthology.orchestra.run.smithi136.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.2 MB/s | 300 kB 00:00 2024-09-16T01:53:12.866 INFO:teuthology.orchestra.run.smithi136.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.3 MB/s | 99 kB 00:00 2024-09-16T01:53:12.890 INFO:teuthology.orchestra.run.smithi136.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-16T01:53:12.969 INFO:teuthology.orchestra.run.smithi172.stdout:(6/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-16T01:53:12.982 INFO:teuthology.orchestra.run.smithi136.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-09-16T01:53:13.027 INFO:teuthology.orchestra.run.smithi172.stdout:(7/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 3.0 MB/s | 2.4 MB 00:00 2024-09-16T01:53:13.074 INFO:teuthology.orchestra.run.smithi136.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 347 kB/s | 32 kB 00:00 2024-09-16T01:53:13.124 INFO:teuthology.orchestra.run.smithi136.stdout:(29/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-16T01:53:13.149 INFO:teuthology.orchestra.run.smithi136.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.9 MB/s | 146 kB 00:00 2024-09-16T01:53:13.311 INFO:teuthology.orchestra.run.smithi172.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 21 MB 00:01 2024-09-16T01:53:13.350 INFO:teuthology.orchestra.run.smithi136.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 16 MB/s | 3.6 MB 00:00 2024-09-16T01:53:13.363 INFO:teuthology.orchestra.run.smithi172.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 505 kB/s | 25 kB 00:00 2024-09-16T01:53:13.413 INFO:teuthology.orchestra.run.smithi136.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 3.9 MB/s | 249 kB 00:00 2024-09-16T01:53:13.530 INFO:teuthology.orchestra.run.smithi136.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 423 kB/s | 49 kB 00:00 2024-09-16T01:53:13.630 INFO:teuthology.orchestra.run.smithi136.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 164 kB/s | 16 kB 00:00 2024-09-16T01:53:13.980 INFO:teuthology.orchestra.run.smithi136.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 8.9 MB/s | 7.4 MB 00:00 2024-09-16T01:53:14.122 INFO:teuthology.orchestra.run.smithi136.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 5.3 MB/s | 765 kB 00:00 2024-09-16T01:53:14.172 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:53:14.173 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:53:14.184 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T01:53:14.185 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T01:53:14.186 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T01:53:14.187 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T01:53:14.188 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T01:53:14.190 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T01:53:14.191 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T01:53:14.214 INFO:teuthology.orchestra.run.smithi136.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 199 kB/s | 18 kB 00:00 2024-09-16T01:53:14.257 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-16T01:53:14.274 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-16T01:53:14.274 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-16T01:53:14.274 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-16T01:53:14.274 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-16T01:53:14.274 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T01:53:14.274 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T01:53:14.274 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T01:53:14.274 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T01:53:14.274 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T01:53:14.275 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T01:53:14.276 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T01:53:14.277 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T01:53:14.278 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T01:53:14.279 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T01:53:14.280 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T01:53:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T01:53:14.282 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T01:53:14.283 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T01:53:14.284 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T01:53:14.285 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T01:53:14.286 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T01:53:14.287 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T01:53:14.287 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-09-16T01:53:14.287 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T01:53:14.287 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:53:14.287 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-16T01:53:14.287 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-16T01:53:14.287 INFO:teuthology.orchestra.run.smithi103.stdout:Install 148 Packages 2024-09-16T01:53:14.287 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 8 Packages 2024-09-16T01:53:14.288 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:53:14.288 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 226 M 2024-09-16T01:53:14.288 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-16T01:53:14.348 INFO:teuthology.orchestra.run.smithi136.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 599 kB/s | 80 kB 00:00 2024-09-16T01:53:14.778 INFO:teuthology.orchestra.run.smithi103.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 34 kB/s | 6.5 kB 00:00 2024-09-16T01:53:14.870 INFO:teuthology.orchestra.run.smithi172.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 8.9 MB/s | 17 MB 00:01 2024-09-16T01:53:14.898 INFO:teuthology.orchestra.run.smithi136.stdout:(39/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 15 MB/s | 49 MB 00:03 2024-09-16T01:53:14.924 INFO:teuthology.orchestra.run.smithi136.stdout:(40/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 220 kB/s | 285 kB 00:01 2024-09-16T01:53:14.957 INFO:teuthology.orchestra.run.smithi136.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-16T01:53:14.961 INFO:teuthology.orchestra.run.smithi103.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 4.8 MB/s | 896 kB 00:00 2024-09-16T01:53:14.987 INFO:teuthology.orchestra.run.smithi172.stdout:(11/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 5.4 MB/s | 11 MB 00:01 2024-09-16T01:53:15.012 INFO:teuthology.orchestra.run.smithi172.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.6 MB/s | 228 kB 00:00 2024-09-16T01:53:15.020 INFO:teuthology.orchestra.run.smithi103.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 2.4 MB/s | 145 kB 00:00 2024-09-16T01:53:15.037 INFO:teuthology.orchestra.run.smithi172.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 542 kB/s | 27 kB 00:00 2024-09-16T01:53:15.062 INFO:teuthology.orchestra.run.smithi172.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 634 kB/s | 32 kB 00:00 2024-09-16T01:53:15.099 INFO:teuthology.orchestra.run.smithi136.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-16T01:53:15.124 INFO:teuthology.orchestra.run.smithi136.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 848 kB/s | 191 kB 00:00 2024-09-16T01:53:15.146 INFO:teuthology.orchestra.run.smithi172.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 1.9 MB/s | 163 kB 00:00 2024-09-16T01:53:15.166 INFO:teuthology.orchestra.run.smithi136.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-09-16T01:53:15.191 INFO:teuthology.orchestra.run.smithi136.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-16T01:53:15.217 INFO:teuthology.orchestra.run.smithi136.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-16T01:53:15.238 INFO:teuthology.orchestra.run.smithi172.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.4 MB/s | 127 kB 00:00 2024-09-16T01:53:15.242 INFO:teuthology.orchestra.run.smithi136.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-16T01:53:15.245 INFO:teuthology.orchestra.run.smithi103.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 8.2 MB/s | 5.4 MB 00:00 2024-09-16T01:53:15.263 INFO:teuthology.orchestra.run.smithi172.stdout:(17/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 3.2 MB/s | 751 kB 00:00 2024-09-16T01:53:15.293 INFO:teuthology.orchestra.run.smithi136.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-09-16T01:53:15.326 INFO:teuthology.orchestra.run.smithi103.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 7.9 MB/s | 2.4 MB 00:00 2024-09-16T01:53:15.380 INFO:teuthology.orchestra.run.smithi172.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 3.5 MB/s | 506 kB 00:00 2024-09-16T01:53:15.393 INFO:teuthology.orchestra.run.smithi136.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-09-16T01:53:15.421 INFO:teuthology.orchestra.run.smithi103.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-16T01:53:15.426 INFO:teuthology.orchestra.run.smithi136.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-16T01:53:15.452 INFO:teuthology.orchestra.run.smithi136.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-16T01:53:15.485 INFO:teuthology.orchestra.run.smithi136.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-16T01:53:15.519 INFO:teuthology.orchestra.run.smithi136.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-16T01:53:15.544 INFO:teuthology.orchestra.run.smithi136.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 355 kB/s | 33 kB 00:00 2024-09-16T01:53:15.586 INFO:teuthology.orchestra.run.smithi136.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-09-16T01:53:15.612 INFO:teuthology.orchestra.run.smithi136.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-09-16T01:53:15.637 INFO:teuthology.orchestra.run.smithi136.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-16T01:53:15.662 INFO:teuthology.orchestra.run.smithi136.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-16T01:53:15.696 INFO:teuthology.orchestra.run.smithi136.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-16T01:53:15.738 INFO:teuthology.orchestra.run.smithi103.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-16T01:53:15.746 INFO:teuthology.orchestra.run.smithi136.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-16T01:53:15.771 INFO:teuthology.orchestra.run.smithi136.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-16T01:53:15.797 INFO:teuthology.orchestra.run.smithi136.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-16T01:53:15.830 INFO:teuthology.orchestra.run.smithi136.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-16T01:53:15.839 INFO:teuthology.orchestra.run.smithi172.stdout:(19/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 20 MB/s | 49 MB 00:02 2024-09-16T01:53:15.856 INFO:teuthology.orchestra.run.smithi136.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.9 MB/s | 581 kB 00:00 2024-09-16T01:53:15.864 INFO:teuthology.orchestra.run.smithi172.stdout:(20/156): python3-ceph-argparse-19.1.1-326.g892 93 kB/s | 45 kB 00:00 2024-09-16T01:53:15.881 INFO:teuthology.orchestra.run.smithi136.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-16T01:53:15.906 INFO:teuthology.orchestra.run.smithi136.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-16T01:53:15.947 INFO:teuthology.orchestra.run.smithi172.stdout:(21/156): python3-ceph-common-19.1.1-326.g8921c 1.2 MB/s | 136 kB 00:00 2024-09-16T01:53:15.973 INFO:teuthology.orchestra.run.smithi172.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.4 MB/s | 161 kB 00:00 2024-09-16T01:53:15.990 INFO:teuthology.orchestra.run.smithi136.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-16T01:53:16.031 INFO:teuthology.orchestra.run.smithi172.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 3.7 MB/s | 320 kB 00:00 2024-09-16T01:53:16.057 INFO:teuthology.orchestra.run.smithi136.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-16T01:53:16.057 INFO:teuthology.orchestra.run.smithi172.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.5 MB/s | 300 kB 00:00 2024-09-16T01:53:16.090 INFO:teuthology.orchestra.run.smithi136.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-16T01:53:16.098 INFO:teuthology.orchestra.run.smithi172.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.5 MB/s | 99 kB 00:00 2024-09-16T01:53:16.115 INFO:teuthology.orchestra.run.smithi136.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-16T01:53:16.124 INFO:teuthology.orchestra.run.smithi172.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-16T01:53:16.172 INFO:teuthology.orchestra.run.smithi103.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 13 MB/s | 21 MB 00:01 2024-09-16T01:53:16.182 INFO:teuthology.orchestra.run.smithi136.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 15 MB/s | 4.9 MB 00:00 2024-09-16T01:53:16.183 INFO:teuthology.orchestra.run.smithi172.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-16T01:53:16.222 INFO:teuthology.orchestra.run.smithi103.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 510 kB/s | 25 kB 00:00 2024-09-16T01:53:16.224 INFO:teuthology.orchestra.run.smithi136.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-16T01:53:16.250 INFO:teuthology.orchestra.run.smithi136.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-16T01:53:16.266 INFO:teuthology.orchestra.run.smithi172.stdout:(28/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 5.3 MB/s | 5.3 MB 00:01 2024-09-16T01:53:16.292 INFO:teuthology.orchestra.run.smithi172.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 294 kB/s | 32 kB 00:00 2024-09-16T01:53:16.334 INFO:teuthology.orchestra.run.smithi136.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-09-16T01:53:16.358 INFO:teuthology.orchestra.run.smithi172.stdout:(30/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-16T01:53:16.359 INFO:teuthology.orchestra.run.smithi136.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 565 kB/s | 61 kB 00:00 2024-09-16T01:53:16.383 INFO:teuthology.orchestra.run.smithi172.stdout:(31/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.2 MB/s | 146 kB 00:00 2024-09-16T01:53:16.384 INFO:teuthology.orchestra.run.smithi136.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-16T01:53:16.409 INFO:teuthology.orchestra.run.smithi136.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-16T01:53:16.434 INFO:teuthology.orchestra.run.smithi136.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-16T01:53:16.460 INFO:teuthology.orchestra.run.smithi136.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-16T01:53:16.467 INFO:teuthology.orchestra.run.smithi172.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 2.9 MB/s | 249 kB 00:00 2024-09-16T01:53:16.485 INFO:teuthology.orchestra.run.smithi136.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-16T01:53:16.510 INFO:teuthology.orchestra.run.smithi136.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-16T01:53:16.522 INFO:teuthology.orchestra.run.smithi103.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 15 MB/s | 17 MB 00:01 2024-09-16T01:53:16.551 INFO:teuthology.orchestra.run.smithi172.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 592 kB/s | 49 kB 00:00 2024-09-16T01:53:16.586 INFO:teuthology.orchestra.run.smithi136.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-16T01:53:16.611 INFO:teuthology.orchestra.run.smithi136.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-16T01:53:16.626 INFO:teuthology.orchestra.run.smithi172.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 220 kB/s | 16 kB 00:00 2024-09-16T01:53:16.636 INFO:teuthology.orchestra.run.smithi136.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-16T01:53:16.653 INFO:teuthology.orchestra.run.smithi136.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-09-16T01:53:16.676 INFO:teuthology.orchestra.run.smithi172.stdout:(35/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 9.3 MB/s | 3.6 MB 00:00 2024-09-16T01:53:16.681 INFO:teuthology.orchestra.run.smithi103.stdout:(11/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 11 MB/s | 11 MB 00:00 2024-09-16T01:53:16.687 INFO:teuthology.orchestra.run.smithi136.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-16T01:53:16.701 INFO:teuthology.orchestra.run.smithi172.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 3.7 MB/s | 285 kB 00:00 2024-09-16T01:53:16.706 INFO:teuthology.orchestra.run.smithi103.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.2 MB/s | 228 kB 00:00 2024-09-16T01:53:16.712 INFO:teuthology.orchestra.run.smithi136.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-16T01:53:16.731 INFO:teuthology.orchestra.run.smithi103.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 541 kB/s | 27 kB 00:00 2024-09-16T01:53:16.737 INFO:teuthology.orchestra.run.smithi136.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T01:53:16.757 INFO:teuthology.orchestra.run.smithi103.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 637 kB/s | 32 kB 00:00 2024-09-16T01:53:16.763 INFO:teuthology.orchestra.run.smithi136.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-16T01:53:16.788 INFO:teuthology.orchestra.run.smithi136.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T01:53:16.802 INFO:teuthology.orchestra.run.smithi172.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-16T01:53:16.807 INFO:teuthology.orchestra.run.smithi103.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 9.8 MB/s | 751 kB 00:00 2024-09-16T01:53:16.813 INFO:teuthology.orchestra.run.smithi136.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-16T01:53:16.827 INFO:teuthology.orchestra.run.smithi172.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 146 kB/s | 18 kB 00:00 2024-09-16T01:53:16.832 INFO:teuthology.orchestra.run.smithi103.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.1 MB/s | 163 kB 00:00 2024-09-16T01:53:16.852 INFO:teuthology.orchestra.run.smithi172.stdout:(39/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 4.3 MB/s | 765 kB 00:00 2024-09-16T01:53:16.866 INFO:teuthology.orchestra.run.smithi103.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.1 MB/s | 127 kB 00:00 2024-09-16T01:53:16.880 INFO:teuthology.orchestra.run.smithi136.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-16T01:53:16.900 INFO:teuthology.orchestra.run.smithi103.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 7.4 MB/s | 506 kB 00:00 2024-09-16T01:53:16.906 INFO:teuthology.orchestra.run.smithi136.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-09-16T01:53:16.939 INFO:teuthology.orchestra.run.smithi136.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-09-16T01:53:16.958 INFO:teuthology.orchestra.run.smithi103.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 775 kB/s | 45 kB 00:00 2024-09-16T01:53:16.961 INFO:teuthology.orchestra.run.smithi172.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-16T01:53:16.965 INFO:teuthology.orchestra.run.smithi136.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-16T01:53:16.990 INFO:teuthology.orchestra.run.smithi136.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-16T01:53:17.011 INFO:teuthology.orchestra.run.smithi172.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-16T01:53:17.015 INFO:teuthology.orchestra.run.smithi136.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-16T01:53:17.034 INFO:teuthology.orchestra.run.smithi103.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.8 MB/s | 136 kB 00:00 2024-09-16T01:53:17.040 INFO:teuthology.orchestra.run.smithi136.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-16T01:53:17.053 INFO:teuthology.orchestra.run.smithi172.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-16T01:53:17.066 INFO:teuthology.orchestra.run.smithi136.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T01:53:17.087 INFO:teuthology.orchestra.run.smithi172.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-16T01:53:17.091 INFO:teuthology.orchestra.run.smithi136.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.8 MB/s | 96 kB 00:00 2024-09-16T01:53:17.112 INFO:teuthology.orchestra.run.smithi172.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-16T01:53:17.116 INFO:teuthology.orchestra.run.smithi136.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T01:53:17.117 INFO:teuthology.orchestra.run.smithi103.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.9 MB/s | 161 kB 00:00 2024-09-16T01:53:17.142 INFO:teuthology.orchestra.run.smithi136.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-16T01:53:17.145 INFO:teuthology.orchestra.run.smithi172.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-09-16T01:53:17.171 INFO:teuthology.orchestra.run.smithi172.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-16T01:53:17.196 INFO:teuthology.orchestra.run.smithi172.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-16T01:53:17.201 INFO:teuthology.orchestra.run.smithi103.stdout:(22/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-16T01:53:17.226 INFO:teuthology.orchestra.run.smithi103.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 2.9 MB/s | 320 kB 00:00 2024-09-16T01:53:17.230 INFO:teuthology.orchestra.run.smithi172.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-16T01:53:17.251 INFO:teuthology.orchestra.run.smithi103.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.9 MB/s | 300 kB 00:00 2024-09-16T01:53:17.263 INFO:teuthology.orchestra.run.smithi172.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 7.3 MB/s | 689 kB 00:00 2024-09-16T01:53:17.277 INFO:teuthology.orchestra.run.smithi103.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T01:53:17.289 INFO:teuthology.orchestra.run.smithi172.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-09-16T01:53:17.302 INFO:teuthology.orchestra.run.smithi103.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-16T01:53:17.314 INFO:teuthology.orchestra.run.smithi172.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-16T01:53:17.339 INFO:teuthology.orchestra.run.smithi172.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 981 kB/s | 107 kB 00:00 2024-09-16T01:53:17.361 INFO:teuthology.orchestra.run.smithi103.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-16T01:53:17.373 INFO:teuthology.orchestra.run.smithi172.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-16T01:53:17.398 INFO:teuthology.orchestra.run.smithi172.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-16T01:53:17.415 INFO:teuthology.orchestra.run.smithi172.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-09-16T01:53:17.417 INFO:teuthology.orchestra.run.smithi136.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-16T01:53:17.428 INFO:teuthology.orchestra.run.smithi103.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-16T01:53:17.432 INFO:teuthology.orchestra.run.smithi172.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-09-16T01:53:17.442 INFO:teuthology.orchestra.run.smithi136.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 68 kB/s | 22 kB 00:00 2024-09-16T01:53:17.445 INFO:teuthology.orchestra.run.smithi103.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 380 kB/s | 32 kB 00:00 2024-09-16T01:53:17.479 INFO:teuthology.orchestra.run.smithi103.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.8 MB/s | 146 kB 00:00 2024-09-16T01:53:17.490 INFO:teuthology.orchestra.run.smithi172.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-09-16T01:53:17.501 INFO:teuthology.orchestra.run.smithi136.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-16T01:53:17.516 INFO:teuthology.orchestra.run.smithi172.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 540 kB/s | 45 kB 00:00 2024-09-16T01:53:17.526 INFO:teuthology.orchestra.run.smithi136.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-16T01:53:17.541 INFO:teuthology.orchestra.run.smithi172.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-16T01:53:17.552 INFO:teuthology.orchestra.run.smithi136.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-16T01:53:17.577 INFO:teuthology.orchestra.run.smithi136.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T01:53:17.591 INFO:teuthology.orchestra.run.smithi172.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-16T01:53:17.602 INFO:teuthology.orchestra.run.smithi136.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-16T01:53:17.617 INFO:teuthology.orchestra.run.smithi172.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 326 kB/s | 24 kB 00:00 2024-09-16T01:53:17.627 INFO:teuthology.orchestra.run.smithi136.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T01:53:17.642 INFO:teuthology.orchestra.run.smithi172.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-16T01:53:17.653 INFO:teuthology.orchestra.run.smithi136.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-09-16T01:53:17.676 INFO:teuthology.orchestra.run.smithi172.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 3.6 MB/s | 581 kB 00:00 2024-09-16T01:53:17.678 INFO:teuthology.orchestra.run.smithi136.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-16T01:53:17.701 INFO:teuthology.orchestra.run.smithi172.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-16T01:53:17.726 INFO:teuthology.orchestra.run.smithi172.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-16T01:53:17.745 INFO:teuthology.orchestra.run.smithi136.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.0 MB/s | 548 kB 00:00 2024-09-16T01:53:17.751 INFO:teuthology.orchestra.run.smithi172.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 510 kB/s | 38 kB 00:00 2024-09-16T01:53:17.770 INFO:teuthology.orchestra.run.smithi136.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T01:53:17.796 INFO:teuthology.orchestra.run.smithi136.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.2 MB/s | 32 kB 00:00 2024-09-16T01:53:17.818 INFO:teuthology.orchestra.run.smithi172.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-16T01:53:17.821 INFO:teuthology.orchestra.run.smithi136.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-16T01:53:17.838 INFO:teuthology.orchestra.run.smithi103.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 21 MB/s | 7.4 MB 00:00 2024-09-16T01:53:17.863 INFO:teuthology.orchestra.run.smithi136.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-16T01:53:17.888 INFO:teuthology.orchestra.run.smithi103.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 4.9 MB/s | 249 kB 00:00 2024-09-16T01:53:17.889 INFO:teuthology.orchestra.run.smithi136.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 839 kB 00:00 2024-09-16T01:53:17.894 INFO:teuthology.orchestra.run.smithi172.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-16T01:53:17.947 INFO:teuthology.orchestra.run.smithi136.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-16T01:53:17.952 INFO:teuthology.orchestra.run.smithi172.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-16T01:53:17.955 INFO:teuthology.orchestra.run.smithi103.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 741 kB/s | 49 kB 00:00 2024-09-16T01:53:17.972 INFO:teuthology.orchestra.run.smithi136.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-09-16T01:53:17.977 INFO:teuthology.orchestra.run.smithi172.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 5.6 MB/s | 910 kB 00:00 2024-09-16T01:53:17.997 INFO:teuthology.orchestra.run.smithi136.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-09-16T01:53:18.011 INFO:teuthology.orchestra.run.smithi172.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-16T01:53:18.013 INFO:teuthology.orchestra.run.smithi103.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 282 kB/s | 16 kB 00:00 2024-09-16T01:53:18.023 INFO:teuthology.orchestra.run.smithi136.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-16T01:53:18.048 INFO:teuthology.orchestra.run.smithi136.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-16T01:53:18.053 INFO:teuthology.orchestra.run.smithi172.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-16T01:53:18.073 INFO:teuthology.orchestra.run.smithi136.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-16T01:53:18.078 INFO:teuthology.orchestra.run.smithi172.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-16T01:53:18.081 INFO:teuthology.orchestra.run.smithi103.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.2 MB/s | 285 kB 00:00 2024-09-16T01:53:18.099 INFO:teuthology.orchestra.run.smithi136.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-16T01:53:18.120 INFO:teuthology.orchestra.run.smithi172.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-16T01:53:18.124 INFO:teuthology.orchestra.run.smithi136.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-09-16T01:53:18.149 INFO:teuthology.orchestra.run.smithi136.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-16T01:53:18.173 INFO:teuthology.orchestra.run.smithi103.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 8.2 MB/s | 765 kB 00:00 2024-09-16T01:53:18.175 INFO:teuthology.orchestra.run.smithi136.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 389 kB/s | 20 kB 00:00 2024-09-16T01:53:18.195 INFO:teuthology.orchestra.run.smithi172.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-09-16T01:53:18.200 INFO:teuthology.orchestra.run.smithi136.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T01:53:18.221 INFO:teuthology.orchestra.run.smithi172.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-16T01:53:18.234 INFO:teuthology.orchestra.run.smithi136.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-09-16T01:53:18.246 INFO:teuthology.orchestra.run.smithi172.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-16T01:53:18.259 INFO:teuthology.orchestra.run.smithi136.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-16T01:53:18.271 INFO:teuthology.orchestra.run.smithi172.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-16T01:53:18.284 INFO:teuthology.orchestra.run.smithi136.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-16T01:53:18.296 INFO:teuthology.orchestra.run.smithi172.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-16T01:53:18.330 INFO:teuthology.orchestra.run.smithi172.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-16T01:53:18.355 INFO:teuthology.orchestra.run.smithi172.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-09-16T01:53:18.377 INFO:teuthology.orchestra.run.smithi136.stdout:(133/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.0 MB/s | 2.0 MB 00:00 2024-09-16T01:53:18.431 INFO:teuthology.orchestra.run.smithi172.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-16T01:53:18.456 INFO:teuthology.orchestra.run.smithi172.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-16T01:53:18.469 INFO:teuthology.orchestra.run.smithi136.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 89 kB/s | 16 kB 00:00 2024-09-16T01:53:18.481 INFO:teuthology.orchestra.run.smithi172.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-16T01:53:18.494 INFO:teuthology.orchestra.run.smithi136.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 774 kB/s | 90 kB 00:00 2024-09-16T01:53:18.507 INFO:teuthology.orchestra.run.smithi172.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T01:53:18.510 INFO:teuthology.orchestra.run.smithi136.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 677 kB/s | 28 kB 00:00 2024-09-16T01:53:18.532 INFO:teuthology.orchestra.run.smithi172.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-16T01:53:18.535 INFO:teuthology.orchestra.run.smithi136.stdout:(137/156): python3-pecan-1.4.2-3.el9.noarch.rpm 987 kB/s | 272 kB 00:00 2024-09-16T01:53:18.552 INFO:teuthology.orchestra.run.smithi136.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 736 kB/s | 31 kB 00:00 2024-09-16T01:53:18.557 INFO:teuthology.orchestra.run.smithi172.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-16T01:53:18.569 INFO:teuthology.orchestra.run.smithi136.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-16T01:53:18.582 INFO:teuthology.orchestra.run.smithi172.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T01:53:18.586 INFO:teuthology.orchestra.run.smithi136.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-16T01:53:18.608 INFO:teuthology.orchestra.run.smithi172.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-16T01:53:18.611 INFO:teuthology.orchestra.run.smithi136.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-16T01:53:18.633 INFO:teuthology.orchestra.run.smithi172.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T01:53:18.637 INFO:teuthology.orchestra.run.smithi136.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-16T01:53:18.648 INFO:teuthology.orchestra.run.smithi103.stdout:(37/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 20 MB/s | 49 MB 00:02 2024-09-16T01:53:18.658 INFO:teuthology.orchestra.run.smithi172.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-16T01:53:18.662 INFO:teuthology.orchestra.run.smithi136.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-16T01:53:18.673 INFO:teuthology.orchestra.run.smithi103.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 36 kB/s | 18 kB 00:00 2024-09-16T01:53:18.687 INFO:teuthology.orchestra.run.smithi136.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-16T01:53:18.713 INFO:teuthology.orchestra.run.smithi136.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-16T01:53:18.738 INFO:teuthology.orchestra.run.smithi136.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T01:53:18.742 INFO:teuthology.orchestra.run.smithi172.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-16T01:53:18.759 INFO:teuthology.orchestra.run.smithi172.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-16T01:53:18.765 INFO:teuthology.orchestra.run.smithi103.stdout:(39/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 2.7 MB/s | 3.6 MB 00:01 2024-09-16T01:53:18.772 INFO:teuthology.orchestra.run.smithi136.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-16T01:53:18.784 INFO:teuthology.orchestra.run.smithi172.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-16T01:53:18.809 INFO:teuthology.orchestra.run.smithi172.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-16T01:53:18.822 INFO:teuthology.orchestra.run.smithi136.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T01:53:18.826 INFO:teuthology.orchestra.run.smithi172.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 854 kB/s | 36 kB 00:00 2024-09-16T01:53:18.841 INFO:teuthology.orchestra.run.smithi103.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 416 kB/s | 80 kB 00:00 2024-09-16T01:53:18.843 INFO:teuthology.orchestra.run.smithi172.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-16T01:53:18.878 INFO:teuthology.orchestra.run.smithi172.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-16T01:53:18.894 INFO:teuthology.orchestra.run.smithi172.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T01:53:18.919 INFO:teuthology.orchestra.run.smithi172.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-16T01:53:18.933 INFO:teuthology.orchestra.run.smithi103.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-16T01:53:18.936 INFO:teuthology.orchestra.run.smithi172.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-16T01:53:18.953 INFO:teuthology.orchestra.run.smithi172.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-16T01:53:18.958 INFO:teuthology.orchestra.run.smithi103.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-16T01:53:19.008 INFO:teuthology.orchestra.run.smithi103.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-16T01:53:19.033 INFO:teuthology.orchestra.run.smithi103.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-16T01:53:19.059 INFO:teuthology.orchestra.run.smithi103.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-16T01:53:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-16T01:53:19.122 INFO:teuthology.orchestra.run.smithi136.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 7.9 MB/s | 2.3 MB 00:00 2024-09-16T01:53:19.126 INFO:teuthology.orchestra.run.smithi103.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-09-16T01:53:19.151 INFO:teuthology.orchestra.run.smithi103.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-16T01:53:19.153 INFO:teuthology.orchestra.run.smithi172.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-16T01:53:19.178 INFO:teuthology.orchestra.run.smithi172.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-09-16T01:53:19.189 INFO:teuthology.orchestra.run.smithi136.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 7.6 MB/s | 3.2 MB 00:00 2024-09-16T01:53:19.245 INFO:teuthology.orchestra.run.smithi172.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-16T01:53:19.248 INFO:teuthology.orchestra.run.smithi136.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 699 kB/s | 41 kB 00:00 2024-09-16T01:53:19.270 INFO:teuthology.orchestra.run.smithi172.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 787 kB/s | 19 kB 00:00 2024-09-16T01:53:19.313 INFO:teuthology.orchestra.run.smithi172.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-16T01:53:19.338 INFO:teuthology.orchestra.run.smithi172.stdout:(108/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.9 MB/s | 308 kB 00:00 2024-09-16T01:53:19.348 INFO:teuthology.orchestra.run.smithi136.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 8.0 MB/s | 1.8 MB 00:00 2024-09-16T01:53:19.380 INFO:teuthology.orchestra.run.smithi172.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-16T01:53:19.390 INFO:teuthology.orchestra.run.smithi136.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 1.7 MB/s | 249 kB 00:00 2024-09-16T01:53:19.405 INFO:teuthology.orchestra.run.smithi172.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-09-16T01:53:19.447 INFO:teuthology.orchestra.run.smithi172.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-16T01:53:19.481 INFO:teuthology.orchestra.run.smithi172.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-16T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 8.1 MB/s | 1.4 MB 00:00 2024-09-16T01:53:19.539 INFO:teuthology.orchestra.run.smithi172.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-16T01:53:19.590 INFO:teuthology.orchestra.run.smithi172.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-09-16T01:53:19.607 INFO:teuthology.orchestra.run.smithi172.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 648 kB/s | 43 kB 00:00 2024-09-16T01:53:19.610 INFO:teuthology.orchestra.run.smithi103.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 233 kB/s | 107 kB 00:00 2024-09-16T01:53:19.632 INFO:teuthology.orchestra.run.smithi172.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-09-16T01:53:19.657 INFO:teuthology.orchestra.run.smithi172.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-16T01:53:19.682 INFO:teuthology.orchestra.run.smithi172.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-16T01:53:19.691 INFO:teuthology.orchestra.run.smithi136.stdout:(155/156): librados2-19.1.1-326.g8921c8ef.el9.x 3.6 MB/s | 3.4 MB 00:00 2024-09-16T01:53:19.727 INFO:teuthology.orchestra.run.smithi103.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-16T01:53:19.733 INFO:teuthology.orchestra.run.smithi172.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-16T01:53:19.758 INFO:teuthology.orchestra.run.smithi172.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-16T01:53:19.792 INFO:teuthology.orchestra.run.smithi172.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-16T01:53:19.824 INFO:teuthology.orchestra.run.smithi136.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.5 MB/s | 2.4 MB 00:00 2024-09-16T01:53:19.829 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:53:19.830 INFO:teuthology.orchestra.run.smithi136.stdout:Total 23 MB/s | 226 MB 00:09 2024-09-16T01:53:19.867 INFO:teuthology.orchestra.run.smithi172.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.4 MB/s | 4.4 MB 00:00 2024-09-16T01:53:19.892 INFO:teuthology.orchestra.run.smithi172.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-09-16T01:53:19.910 INFO:teuthology.orchestra.run.smithi103.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 843 kB/s | 689 kB 00:00 2024-09-16T01:53:19.917 INFO:teuthology.orchestra.run.smithi172.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-16T01:53:19.943 INFO:teuthology.orchestra.run.smithi172.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-16T01:53:19.944 INFO:teuthology.orchestra.run.smithi103.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-16T01:53:19.968 INFO:teuthology.orchestra.run.smithi172.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-16T01:53:19.969 INFO:teuthology.orchestra.run.smithi103.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 1.3 MB/s | 313 kB 00:00 2024-09-16T01:53:19.993 INFO:teuthology.orchestra.run.smithi172.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-16T01:53:20.018 INFO:teuthology.orchestra.run.smithi172.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-16T01:53:20.020 INFO:teuthology.orchestra.run.smithi103.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 299 kB/s | 33 kB 00:00 2024-09-16T01:53:20.044 INFO:teuthology.orchestra.run.smithi172.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T01:53:20.077 INFO:teuthology.orchestra.run.smithi172.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-16T01:53:20.079 INFO:teuthology.orchestra.run.smithi103.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 167 kB/s | 18 kB 00:00 2024-09-16T01:53:20.103 INFO:teuthology.orchestra.run.smithi172.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-16T01:53:20.129 INFO:teuthology.orchestra.run.smithi103.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-09-16T01:53:20.153 INFO:teuthology.orchestra.run.smithi172.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-16T01:53:20.195 INFO:teuthology.orchestra.run.smithi172.stdout:(133/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.7 MB/s | 2.0 MB 00:00 2024-09-16T01:53:20.196 INFO:teuthology.orchestra.run.smithi103.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-16T01:53:20.220 INFO:teuthology.orchestra.run.smithi172.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-09-16T01:53:20.238 INFO:teuthology.orchestra.run.smithi103.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-09-16T01:53:20.245 INFO:teuthology.orchestra.run.smithi172.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-16T01:53:20.271 INFO:teuthology.orchestra.run.smithi172.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-16T01:53:20.288 INFO:teuthology.orchestra.run.smithi103.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-09-16T01:53:20.296 INFO:teuthology.orchestra.run.smithi172.stdout:(137/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-09-16T01:53:20.321 INFO:teuthology.orchestra.run.smithi172.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-09-16T01:53:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-16T01:53:20.371 INFO:teuthology.orchestra.run.smithi172.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-16T01:53:20.397 INFO:teuthology.orchestra.run.smithi103.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 226 kB/s | 24 kB 00:00 2024-09-16T01:53:20.397 INFO:teuthology.orchestra.run.smithi172.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-16T01:53:20.422 INFO:teuthology.orchestra.run.smithi172.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-16T01:53:20.447 INFO:teuthology.orchestra.run.smithi172.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-16T01:53:20.473 INFO:teuthology.orchestra.run.smithi172.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-16T01:53:20.498 INFO:teuthology.orchestra.run.smithi172.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-16T01:53:20.506 INFO:teuthology.orchestra.run.smithi103.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-16T01:53:20.523 INFO:teuthology.orchestra.run.smithi172.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T01:53:20.531 INFO:teuthology.orchestra.run.smithi103.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 1.9 MB/s | 581 kB 00:00 2024-09-16T01:53:20.549 INFO:teuthology.orchestra.run.smithi172.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-16T01:53:20.564 INFO:teuthology.orchestra.run.smithi103.stdout:(63/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.8 MB/s | 662 kB 00:00 2024-09-16T01:53:20.607 INFO:teuthology.orchestra.run.smithi172.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T01:53:20.623 INFO:teuthology.orchestra.run.smithi103.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-16T01:53:20.649 INFO:teuthology.orchestra.run.smithi103.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-16T01:53:20.674 INFO:teuthology.orchestra.run.smithi103.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 353 kB/s | 38 kB 00:00 2024-09-16T01:53:20.683 INFO:teuthology.orchestra.run.smithi172.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 31 MB/s | 2.3 MB 00:00 2024-09-16T01:53:20.750 INFO:teuthology.orchestra.run.smithi172.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-16T01:53:20.800 INFO:teuthology.orchestra.run.smithi172.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 818 kB/s | 41 kB 00:00 2024-09-16T01:53:20.850 INFO:teuthology.orchestra.run.smithi172.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-16T01:53:20.899 INFO:teuthology.orchestra.run.smithi103.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 7.1 MB/s | 1.6 MB 00:00 2024-09-16T01:53:20.917 INFO:teuthology.orchestra.run.smithi172.stdout:(153/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 8.6 MB/s | 3.2 MB 00:00 2024-09-16T01:53:20.967 INFO:teuthology.orchestra.run.smithi172.stdout:(154/156): librados2-19.1.1-326.g8921c8ef.el9.x 7.7 MB/s | 3.4 MB 00:00 2024-09-16T01:53:21.009 INFO:teuthology.orchestra.run.smithi172.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 8.5 MB/s | 1.4 MB 00:00 2024-09-16T01:53:21.051 INFO:teuthology.orchestra.run.smithi172.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-09-16T01:53:21.058 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-09-16T01:53:21.058 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:53:21.058 INFO:teuthology.orchestra.run.smithi172.stdout:Total 23 MB/s | 226 MB 00:09 2024-09-16T01:53:21.116 INFO:teuthology.orchestra.run.smithi103.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.1 MB/s | 910 kB 00:00 2024-09-16T01:53:21.206 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-09-16T01:53:21.206 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-09-16T01:53:21.233 INFO:teuthology.orchestra.run.smithi103.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-16T01:53:22.302 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-16T01:53:22.455 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-16T01:53:22.455 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-16T01:53:22.999 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-09-16T01:53:23.000 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-09-16T01:53:23.191 INFO:teuthology.orchestra.run.smithi103.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 1.7 MB/s | 4.4 MB 00:02 2024-09-16T01:53:24.271 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-16T01:53:24.272 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-16T01:53:24.733 INFO:teuthology.orchestra.run.smithi103.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 1.2 MB/s | 4.9 MB 00:04 2024-09-16T01:53:24.767 INFO:teuthology.orchestra.run.smithi103.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 656 kB/s | 1.0 MB 00:01 2024-09-16T01:53:24.850 INFO:teuthology.orchestra.run.smithi103.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 1.6 MB/s | 5.8 MB 00:03 2024-09-16T01:53:24.876 INFO:teuthology.orchestra.run.smithi103.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 616 kB/s | 87 kB 00:00 2024-09-16T01:53:24.959 INFO:teuthology.orchestra.run.smithi103.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 565 kB/s | 61 kB 00:00 2024-09-16T01:53:25.001 INFO:teuthology.orchestra.run.smithi103.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-16T01:53:25.093 INFO:teuthology.orchestra.run.smithi103.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-16T01:53:25.118 INFO:teuthology.orchestra.run.smithi103.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-09-16T01:53:25.144 INFO:teuthology.orchestra.run.smithi103.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-09-16T01:53:25.202 INFO:teuthology.orchestra.run.smithi103.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-16T01:53:25.245 INFO:teuthology.orchestra.run.smithi103.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-16T01:53:25.270 INFO:teuthology.orchestra.run.smithi103.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 276 kB/s | 35 kB 00:00 2024-09-16T01:53:25.362 INFO:teuthology.orchestra.run.smithi103.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-16T01:53:25.387 INFO:teuthology.orchestra.run.smithi103.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-16T01:53:25.487 INFO:teuthology.orchestra.run.smithi103.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-09-16T01:53:25.513 INFO:teuthology.orchestra.run.smithi103.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-09-16T01:53:25.538 INFO:teuthology.orchestra.run.smithi103.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 438 kB/s | 77 kB 00:00 2024-09-16T01:53:25.605 INFO:teuthology.orchestra.run.smithi103.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-16T01:53:25.630 INFO:teuthology.orchestra.run.smithi103.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-16T01:53:25.664 INFO:teuthology.orchestra.run.smithi103.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-09-16T01:53:25.715 INFO:teuthology.orchestra.run.smithi103.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-09-16T01:53:25.740 INFO:teuthology.orchestra.run.smithi103.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-09-16T01:53:25.823 INFO:teuthology.orchestra.run.smithi103.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-16T01:53:25.857 INFO:teuthology.orchestra.run.smithi103.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 379 kB/s | 44 kB 00:00 2024-09-16T01:53:25.940 INFO:teuthology.orchestra.run.smithi103.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-16T01:53:25.974 INFO:teuthology.orchestra.run.smithi103.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 306 kB/s | 36 kB 00:00 2024-09-16T01:53:26.049 INFO:teuthology.orchestra.run.smithi103.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-16T01:53:26.099 INFO:teuthology.orchestra.run.smithi103.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-16T01:53:26.158 INFO:teuthology.orchestra.run.smithi103.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-09-16T01:53:26.217 INFO:teuthology.orchestra.run.smithi103.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 819 kB/s | 96 kB 00:00 2024-09-16T01:53:26.392 INFO:teuthology.orchestra.run.smithi103.stdout:(101/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-16T01:53:26.417 INFO:teuthology.orchestra.run.smithi103.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 224 kB/s | 45 kB 00:00 2024-09-16T01:53:26.443 INFO:teuthology.orchestra.run.smithi103.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 224 kB/s | 64 kB 00:00 2024-09-16T01:53:26.501 INFO:teuthology.orchestra.run.smithi103.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-16T01:53:26.560 INFO:teuthology.orchestra.run.smithi103.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.9 MB/s | 863 kB 00:00 2024-09-16T01:53:26.611 INFO:teuthology.orchestra.run.smithi103.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 3.2 MB/s | 548 kB 00:00 2024-09-16T01:53:26.661 INFO:teuthology.orchestra.run.smithi103.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 198 kB/s | 19 kB 00:00 2024-09-16T01:53:26.686 INFO:teuthology.orchestra.run.smithi103.stdout:(108/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.6 MB/s | 308 kB 00:00 2024-09-16T01:53:26.711 INFO:teuthology.orchestra.run.smithi103.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 513 kB/s | 26 kB 00:00 2024-09-16T01:53:26.737 INFO:teuthology.orchestra.run.smithi103.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 963 kB/s | 49 kB 00:00 2024-09-16T01:53:26.762 INFO:teuthology.orchestra.run.smithi103.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T01:53:26.787 INFO:teuthology.orchestra.run.smithi103.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-16T01:53:26.854 INFO:teuthology.orchestra.run.smithi103.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-16T01:53:26.888 INFO:teuthology.orchestra.run.smithi103.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-16T01:53:26.981 INFO:teuthology.orchestra.run.smithi103.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-09-16T01:53:27.005 INFO:teuthology.orchestra.run.smithi103.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-09-16T01:53:27.031 INFO:teuthology.orchestra.run.smithi103.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 287 kB/s | 14 kB 00:00 2024-09-16T01:53:27.056 INFO:teuthology.orchestra.run.smithi103.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-16T01:53:27.098 INFO:teuthology.orchestra.run.smithi103.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-16T01:53:27.124 INFO:teuthology.orchestra.run.smithi103.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-16T01:53:27.157 INFO:teuthology.orchestra.run.smithi103.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-16T01:53:27.205 INFO:teuthology.orchestra.run.smithi103.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 3.0 MB/s | 144 kB 00:00 2024-09-16T01:53:27.222 INFO:teuthology.orchestra.run.smithi103.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 654 kB/s | 11 kB 00:00 2024-09-16T01:53:27.247 INFO:teuthology.orchestra.run.smithi103.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-16T01:53:27.322 INFO:teuthology.orchestra.run.smithi103.stdout:(125/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-09-16T01:53:27.348 INFO:teuthology.orchestra.run.smithi103.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-09-16T01:53:27.373 INFO:teuthology.orchestra.run.smithi103.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-16T01:53:27.398 INFO:teuthology.orchestra.run.smithi103.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-16T01:53:27.424 INFO:teuthology.orchestra.run.smithi103.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T01:53:27.449 INFO:teuthology.orchestra.run.smithi103.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T01:53:27.474 INFO:teuthology.orchestra.run.smithi103.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-16T01:53:27.508 INFO:teuthology.orchestra.run.smithi103.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-16T01:53:27.550 INFO:teuthology.orchestra.run.smithi103.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 394 kB/s | 16 kB 00:00 2024-09-16T01:53:27.592 INFO:teuthology.orchestra.run.smithi103.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-16T01:53:27.617 INFO:teuthology.orchestra.run.smithi103.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-16T01:53:27.668 INFO:teuthology.orchestra.run.smithi103.stdout:(136/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-16T01:53:27.693 INFO:teuthology.orchestra.run.smithi103.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-09-16T01:53:27.718 INFO:teuthology.orchestra.run.smithi103.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-09-16T01:53:27.762 INFO:teuthology.orchestra.run.smithi103.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.9 MB/s | 188 kB 00:00 2024-09-16T01:53:27.794 INFO:teuthology.orchestra.run.smithi103.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-09-16T01:53:27.819 INFO:teuthology.orchestra.run.smithi103.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 587 kB/s | 59 kB 00:00 2024-09-16T01:53:27.844 INFO:teuthology.orchestra.run.smithi103.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 916 kB/s | 74 kB 00:00 2024-09-16T01:53:27.869 INFO:teuthology.orchestra.run.smithi103.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-16T01:53:27.895 INFO:teuthology.orchestra.run.smithi103.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-16T01:53:27.920 INFO:teuthology.orchestra.run.smithi103.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-16T01:53:27.954 INFO:teuthology.orchestra.run.smithi103.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-16T01:53:27.979 INFO:teuthology.orchestra.run.smithi103.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-16T01:53:28.030 INFO:teuthology.orchestra.run.smithi103.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T01:53:28.222 INFO:teuthology.orchestra.run.smithi103.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 13 MB/s | 3.2 MB 00:00 2024-09-16T01:53:28.347 INFO:teuthology.orchestra.run.smithi103.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 7.3 MB/s | 2.3 MB 00:00 2024-09-16T01:53:28.398 INFO:teuthology.orchestra.run.smithi103.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 818 kB/s | 41 kB 00:00 2024-09-16T01:53:28.456 INFO:teuthology.orchestra.run.smithi103.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-16T01:53:28.615 INFO:teuthology.orchestra.run.smithi103.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 4.6 MB/s | 1.8 MB 00:00 2024-09-16T01:53:28.656 INFO:teuthology.orchestra.run.smithi103.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 6.8 MB/s | 1.4 MB 00:00 2024-09-16T01:53:28.849 INFO:teuthology.orchestra.run.smithi103.stdout:(155/156): librados2-19.1.1-326.g8921c8ef.el9.x 3.8 MB/s | 3.4 MB 00:00 2024-09-16T01:53:28.949 INFO:teuthology.orchestra.run.smithi103.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.2 MB/s | 2.4 MB 00:00 2024-09-16T01:53:28.954 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:53:28.954 INFO:teuthology.orchestra.run.smithi103.stdout:Total 15 MB/s | 226 MB 00:14 2024-09-16T01:53:29.347 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-09-16T01:53:29.477 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T01:53:29.515 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T01:53:29.577 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T01:53:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T01:53:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T01:53:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T01:53:30.012 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T01:53:30.130 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:53:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:53:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:53:30.371 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:53:30.402 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:53:30.549 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T01:53:30.599 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T01:53:30.613 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-16T01:53:30.627 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:53:30.704 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T01:53:30.730 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:53:30.743 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T01:53:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:53:30.804 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T01:53:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:53:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:53:30.838 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T01:53:30.864 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:53:30.864 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T01:53:30.907 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T01:53:31.002 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-16T01:53:31.055 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:53:31.186 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-16T01:53:31.186 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-16T01:53:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:53:31.237 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T01:53:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T01:53:31.340 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:53:31.431 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:53:31.462 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:53:31.491 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:53:31.530 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:53:31.582 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:53:31.611 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:53:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:53:31.766 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T01:53:31.816 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T01:53:31.845 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:53:31.850 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T01:53:31.946 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T01:53:31.948 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:53:31.979 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:53:31.984 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T01:53:32.019 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T01:53:32.042 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:53:32.071 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:53:32.083 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T01:53:32.097 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:53:32.128 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T01:53:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T01:53:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T01:53:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T01:53:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T01:53:32.281 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:53:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T01:53:32.391 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T01:53:32.419 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:53:32.421 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T01:53:32.466 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T01:53:32.543 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T01:53:32.570 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T01:53:32.585 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T01:53:32.629 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T01:53:32.657 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T01:53:32.681 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:53:32.728 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T01:53:32.755 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:53:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T01:53:32.847 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T01:53:32.982 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:53:32.983 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T01:53:33.050 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T01:53:33.088 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T01:53:33.181 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T01:53:33.218 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T01:53:33.252 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T01:53:33.317 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T01:53:33.362 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T01:53:33.394 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T01:53:33.441 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T01:53:33.473 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T01:53:33.501 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T01:53:33.534 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-16T01:53:33.535 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-16T01:53:33.578 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T01:53:33.589 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T01:53:33.632 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T01:53:33.662 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T01:53:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T01:53:33.697 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T01:53:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T01:53:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T01:53:33.772 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T01:53:33.780 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T01:53:33.808 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T01:53:33.822 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T01:53:33.851 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T01:53:33.880 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T01:53:33.951 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T01:53:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T01:53:33.994 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T01:53:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T01:53:34.069 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T01:53:34.112 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T01:53:34.185 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T01:53:34.205 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T01:53:34.216 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T01:53:34.252 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T01:53:34.264 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T01:53:34.298 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T01:53:34.343 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T01:53:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T01:53:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T01:53:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T01:53:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T01:53:34.655 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T01:53:34.759 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T01:53:34.797 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T01:53:34.876 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T01:53:34.913 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T01:53:34.956 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T01:53:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T01:53:34.994 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T01:53:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T01:53:35.036 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T01:53:35.038 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T01:53:35.063 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T01:53:35.138 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T01:53:35.191 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T01:53:35.263 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T01:53:35.264 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T01:53:35.303 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T01:53:35.319 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T01:53:35.331 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T01:53:35.368 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T01:53:35.400 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T01:53:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T01:53:35.431 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T01:53:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T01:53:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T01:53:35.467 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T01:53:35.501 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T01:53:35.523 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T01:53:35.535 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T01:53:35.558 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T01:53:35.576 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T01:53:35.602 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T01:53:35.610 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T01:53:35.644 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T01:53:35.696 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T01:53:35.741 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T01:53:35.782 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T01:53:35.797 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T01:53:35.845 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T01:53:35.887 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T01:53:35.900 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T01:53:35.957 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T01:53:35.991 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T01:53:36.227 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T01:53:36.270 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T01:53:36.311 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T01:53:36.336 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T01:53:36.429 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T01:53:36.558 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T01:53:36.592 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T01:53:36.620 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T01:53:36.655 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T01:53:36.677 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T01:53:36.702 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T01:53:36.737 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T01:53:36.741 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T01:53:36.772 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T01:53:36.779 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T01:53:36.808 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T01:53:36.841 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T01:53:36.879 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T01:53:36.882 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T01:53:36.917 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T01:53:36.934 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T01:53:36.979 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T01:53:37.005 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T01:53:37.047 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T01:53:37.058 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T01:53:37.078 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T01:53:37.090 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T01:53:37.184 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T01:53:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T01:53:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T01:53:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T01:53:37.382 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T01:53:37.977 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T01:53:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T01:53:38.029 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T01:53:38.069 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T01:53:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T01:53:38.169 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T01:53:38.215 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T01:53:38.260 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T01:53:38.348 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T01:53:38.379 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T01:53:38.568 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T01:53:38.593 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T01:53:38.664 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T01:53:39.332 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T01:53:39.353 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T01:53:39.442 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T01:53:39.443 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T01:53:39.471 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T01:53:39.504 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T01:53:39.813 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T01:53:39.881 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:53:39.975 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:53:40.006 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T01:53:40.035 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-16T01:53:40.043 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T01:53:40.255 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T01:53:40.489 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T01:53:40.524 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T01:53:40.603 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T01:53:40.621 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:53:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T01:53:40.680 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T01:53:40.685 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T01:53:40.693 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:53:40.714 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T01:53:40.741 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T01:53:40.772 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T01:53:40.799 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T01:53:40.833 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T01:53:41.101 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T01:53:41.142 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T01:53:41.179 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:53:41.206 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:53:41.279 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:53:41.309 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T01:53:41.345 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T01:53:41.350 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:53:41.382 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:53:41.486 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:53:41.528 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:53:41.732 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T01:53:41.799 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T01:53:41.827 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T01:53:41.828 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:53:41.857 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:53:41.911 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:53:41.941 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T01:53:41.954 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:53:41.988 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:53:42.070 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:53:42.097 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:53:42.136 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:53:42.349 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:53:42.526 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:53:42.746 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T01:53:42.955 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:53:42.993 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:53:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:53:43.373 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:53:43.542 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:53:43.581 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:53:43.657 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:53:43.673 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:53:43.673 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T01:53:43.673 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T01:53:43.673 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:53:43.707 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:53:43.807 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T01:53:43.924 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T01:53:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T01:53:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T01:53:44.105 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T01:53:44.170 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T01:53:44.208 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T01:53:44.262 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T01:53:44.263 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:53:44.296 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:53:44.301 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T01:53:44.334 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T01:53:44.422 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T01:53:44.478 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T01:53:44.514 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T01:53:44.561 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T01:53:44.674 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T01:53:44.709 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T01:53:44.754 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T01:53:44.782 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T01:53:44.843 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:53:44.879 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T01:53:44.882 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:53:44.934 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T01:53:44.974 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:53:44.974 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T01:53:44.974 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T01:53:44.974 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:53:45.000 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:53:45.025 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T01:53:45.193 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T01:53:45.253 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T01:53:45.882 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T01:53:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T01:53:46.041 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T01:53:46.084 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T01:53:46.123 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T01:53:46.176 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T01:53:46.364 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T01:53:46.452 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T01:53:46.511 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T01:53:46.595 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T01:53:46.636 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T01:53:46.672 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T01:53:46.731 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T01:53:46.772 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T01:53:46.820 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T01:53:46.887 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T01:53:47.059 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T01:53:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T01:53:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T01:53:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T01:53:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T01:53:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T01:53:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T01:53:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T01:53:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T01:53:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T01:53:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T01:53:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T01:53:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T01:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T01:53:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T01:53:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T01:53:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T01:53:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T01:53:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T01:53:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T01:53:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T01:53:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T01:53:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T01:53:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T01:53:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T01:53:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T01:53:49.494 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T01:53:49.528 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T01:53:49.644 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T01:53:49.719 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T01:53:49.773 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T01:53:49.868 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T01:53:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T01:53:50.146 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T01:53:50.186 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T01:53:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T01:53:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T01:53:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T01:53:52.795 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T01:53:52.901 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T01:53:52.937 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T01:53:52.970 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T01:53:53.310 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T01:53:53.357 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:53:53.502 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:53:53.538 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T01:53:53.593 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T01:53:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T01:53:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:53:54.235 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:53:54.262 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T01:53:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:53:55.615 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-09-16T01:53:55.615 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-09-16T01:53:55.615 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-09-16T01:53:55.615 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-09-16T01:53:55.615 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-09-16T01:53:55.616 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-09-16T01:53:55.616 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-09-16T01:53:55.616 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:53:55.779 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:53:55.795 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:53:55.795 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:53:55.795 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T01:53:55.795 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:53:56.529 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:53:56.529 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2024-09-16T01:53:56.529 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2024-09-16T01:53:56.529 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2024-09-16T01:53:56.529 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2024-09-16T01:53:56.529 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2024-09-16T01:53:56.529 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /root 2024-09-16T01:53:56.530 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2024-09-16T01:53:56.530 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:53:56.701 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:53:56.717 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:53:56.717 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:53:56.718 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T01:53:56.718 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:53:56.808 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:53:56.842 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:53:57.272 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:53:57.300 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:53:57.300 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:53:57.300 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T01:53:57.300 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:53:57.301 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:53:57.301 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:53:57.476 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:53:57.530 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:53:57.540 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:53:57.564 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:53:57.564 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:53:57.564 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T01:53:57.564 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:53:57.564 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:53:57.564 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:53:57.647 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:53:57.647 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T01:53:57.647 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T01:53:57.648 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:53:57.689 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:53:58.009 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:53:58.033 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:53:58.033 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:53:58.033 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T01:53:58.033 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:53:58.034 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:53:58.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:53:58.147 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T01:53:58.204 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T01:53:58.205 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:53:58.232 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:53:58.232 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:53:58.232 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T01:53:58.232 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:53:58.232 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:53:58.232 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:53:58.260 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T01:53:58.447 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T01:53:58.456 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:53:58.479 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:53:58.479 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:53:58.479 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T01:53:58.479 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:53:58.480 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:53:58.480 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:53:58.493 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T01:53:58.562 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T01:53:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T01:53:58.790 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T01:53:58.836 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T01:53:58.867 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T01:53:58.895 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T01:53:58.915 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:53:58.939 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:53:58.939 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:53:58.939 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T01:53:58.939 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:53:58.939 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:53:58.939 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:53:59.045 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T01:53:59.102 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T01:53:59.166 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T01:53:59.325 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:53:59.351 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T01:53:59.352 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:53:59.383 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T01:53:59.390 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T01:53:59.446 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T01:53:59.467 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T01:53:59.475 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:53:59.556 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T01:53:59.695 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T01:53:59.740 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T01:53:59.772 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T01:53:59.800 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T01:54:00.245 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:54:00.273 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:54:00.305 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T01:54:00.367 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T01:54:00.396 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:54:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:54:00.552 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:54:01.441 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:54:01.473 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:54:01.514 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:54:01.536 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:54:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:54:01.800 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T01:54:01.838 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:54:01.862 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:54:01.862 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:01.862 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T01:54:01.862 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:54:01.862 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:54:01.862 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:54:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:54:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:54:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T01:54:02.070 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T01:54:02.437 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:54:02.465 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:54:02.566 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:54:02.726 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T01:54:02.767 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:54:02.790 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:54:02.791 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:02.791 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T01:54:02.791 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:54:02.791 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:54:02.791 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:54:02.843 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:54:02.855 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:54:02.951 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T01:54:03.000 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T01:54:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T01:54:03.033 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:54:03.057 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:54:03.057 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:03.057 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T01:54:03.057 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:54:03.058 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:54:03.058 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:54:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:54:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:54:03.124 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:03.124 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T01:54:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:54:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:54:03.200 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:54:03.200 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:03.200 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T01:54:03.200 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:54:03.200 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:54:03.200 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:54:03.502 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:54:03.526 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:54:03.526 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:03.526 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T01:54:03.526 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:54:03.526 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:54:03.527 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:54:03.924 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T01:54:03.953 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:54:03.977 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:54:03.977 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:03.978 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T01:54:03.978 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:54:03.978 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:54:03.978 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:54:04.021 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:54:04.044 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:54:04.044 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:04.044 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T01:54:04.044 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:54:04.096 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:54:04.120 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:54:04.120 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:04.120 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T01:54:04.120 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:54:04.120 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:54:04.120 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:54:04.430 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:54:04.457 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:54:04.457 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:04.457 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T01:54:04.457 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:54:04.457 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:54:04.457 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:54:08.072 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T01:54:08.113 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T01:54:08.157 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T01:54:08.186 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T01:54:08.214 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T01:54:08.399 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T01:54:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T01:54:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T01:54:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T01:54:08.550 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T01:54:09.009 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T01:54:09.050 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T01:54:09.091 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T01:54:09.115 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:54:09.123 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T01:54:09.150 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T01:54:09.157 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:54:09.178 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:54:09.335 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T01:54:09.388 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T01:54:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T01:54:09.448 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T01:54:09.486 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T01:54:10.011 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T01:54:10.096 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T01:54:10.128 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T01:54:10.135 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:54:10.183 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:54:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:54:10.209 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T01:54:10.249 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T01:54:10.274 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T01:54:10.300 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T01:54:10.300 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:54:10.401 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:54:10.402 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:54:10.486 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:54:10.510 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T01:54:10.516 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:54:10.516 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:54:10.610 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:54:10.610 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-09-16T01:54:10.611 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-09-16T01:54:10.611 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-09-16T01:54:10.611 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-09-16T01:54:10.611 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-09-16T01:54:10.611 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-09-16T01:54:10.611 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-09-16T01:54:10.611 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:10.687 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:54:10.687 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:54:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:54:10.839 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:54:10.844 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:54:10.876 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:54:10.876 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:10.876 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T01:54:10.876 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:11.074 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T01:54:11.163 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T01:54:11.204 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T01:54:11.285 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T01:54:11.324 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T01:54:11.349 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T01:54:11.375 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T01:54:11.375 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:54:11.459 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:54:11.459 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:54:11.543 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:54:11.568 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T01:54:11.574 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:54:11.574 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:54:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:54:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:54:11.846 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:54:11.909 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:54:12.451 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:54:12.508 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:54:12.508 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:12.508 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T01:54:12.509 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:54:12.509 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:54:12.509 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:54:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:54:12.826 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:12.826 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T01:54:12.826 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:54:12.826 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:54:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:13.302 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:54:13.352 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:54:13.352 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:13.352 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T01:54:13.352 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:54:13.353 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:54:13.353 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:13.468 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T01:54:13.538 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T01:54:13.601 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T01:54:13.826 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T01:54:13.897 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T01:54:13.985 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T01:54:14.094 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T01:54:14.257 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T01:54:14.322 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T01:54:14.362 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T01:54:14.388 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T01:54:14.955 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:54:14.988 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:54:15.030 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T01:54:15.131 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T01:54:15.169 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:54:16.397 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:54:16.455 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:54:16.989 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:54:16.989 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:54:16.990 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T01:54:16.991 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T01:54:16.992 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T01:54:16.993 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T01:54:16.994 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T01:54:16.995 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T01:54:16.996 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T01:54:16.997 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T01:54:16.998 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T01:54:16.999 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T01:54:17.000 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T01:54:17.001 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T01:54:17.002 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T01:54:17.546 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:54:17.580 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:54:17.674 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:54:17.674 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T01:54:17.674 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T01:54:17.674 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T01:54:17.675 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T01:54:17.675 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T01:54:17.675 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T01:54:17.675 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T01:54:17.675 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:54:17.675 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:54:17.675 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:54:17.675 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T01:54:17.675 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T01:54:17.675 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:54:17.676 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T01:54:17.676 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T01:54:17.676 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T01:54:17.676 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T01:54:17.676 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T01:54:17.676 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T01:54:17.676 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T01:54:17.676 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T01:54:17.677 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T01:54:17.678 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T01:54:17.679 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T01:54:17.680 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T01:54:17.680 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T01:54:17.680 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T01:54:17.680 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T01:54:17.680 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T01:54:17.680 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T01:54:17.680 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T01:54:17.680 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T01:54:17.681 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T01:54:17.681 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T01:54:17.681 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T01:54:17.681 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T01:54:17.681 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T01:54:17.681 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T01:54:17.681 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T01:54:17.681 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T01:54:17.681 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T01:54:17.681 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T01:54:17.682 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T01:54:17.682 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T01:54:17.682 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T01:54:17.682 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T01:54:17.682 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T01:54:17.682 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T01:54:17.682 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T01:54:17.682 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T01:54:17.682 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T01:54:17.682 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T01:54:17.683 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T01:54:17.684 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T01:54:17.685 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T01:54:17.685 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T01:54:17.685 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T01:54:17.685 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T01:54:17.685 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T01:54:17.685 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T01:54:17.685 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T01:54:17.685 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T01:54:17.685 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T01:54:17.686 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T01:54:17.687 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T01:54:17.688 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T01:54:17.689 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T01:54:17.689 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T01:54:17.689 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T01:54:17.689 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T01:54:17.689 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T01:54:17.689 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T01:54:17.689 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T01:54:17.719 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:54:17.912 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T01:54:17.941 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:54:17.992 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:54:17.992 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:17.992 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T01:54:17.992 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:54:17.992 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:54:17.992 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:18.050 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:54:18.076 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:54:18.183 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T01:54:18.252 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T01:54:19.250 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T01:54:19.303 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:54:19.354 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:54:19.354 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:19.354 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T01:54:19.354 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:54:19.354 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:54:19.355 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:19.403 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:54:19.453 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:54:19.453 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:19.453 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T01:54:19.453 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:19.512 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:54:19.563 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:54:19.563 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:19.563 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T01:54:19.564 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:54:19.564 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:54:19.564 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:19.682 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:54:19.682 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:54:19.682 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T01:54:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T01:54:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:19.685 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:19.685 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:19.685 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:19.685 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.685 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.685 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:19.685 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.685 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T01:54:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T01:54:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T01:54:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T01:54:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T01:54:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T01:54:19.692 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T01:54:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T01:54:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T01:54:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T01:54:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T01:54:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T01:54:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T01:54:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T01:54:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T01:54:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:54:19.696 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-09-16T01:54:19.896 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:54:19.948 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:54:19.948 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:54:19.948 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T01:54:19.948 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:54:19.948 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:54:19.948 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:20.138 DEBUG:teuthology.parallel:result is None 2024-09-16T01:54:20.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:54:20.212 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:54:20.212 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T01:54:20.213 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T01:54:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.216 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T01:54:20.217 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T01:54:20.218 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T01:54:20.219 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T01:54:20.220 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T01:54:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T01:54:20.222 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T01:54:20.223 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T01:54:20.224 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T01:54:20.225 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T01:54:20.225 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:54:20.225 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-16T01:54:20.625 DEBUG:teuthology.parallel:result is None 2024-09-16T01:54:24.893 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T01:54:24.935 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T01:54:24.979 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T01:54:25.016 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T01:54:25.052 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T01:54:25.286 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T01:54:25.332 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T01:54:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T01:54:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T01:54:25.429 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T01:54:26.200 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:54:26.245 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:54:26.286 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:54:27.556 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T01:54:27.659 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T01:54:27.716 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T01:54:27.840 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T01:54:27.879 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T01:54:27.904 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T01:54:27.941 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T01:54:27.941 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:54:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:54:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:54:28.183 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:54:28.207 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T01:54:28.222 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:54:28.222 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:54:28.468 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:54:28.468 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:54:28.578 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:54:28.659 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:54:28.853 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:54:28.854 INFO:teuthology.orchestra.run.smithi103.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-16T01:54:28.854 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:34.541 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T01:54:34.542 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T01:54:34.542 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T01:54:34.542 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T01:54:34.542 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T01:54:34.542 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T01:54:34.542 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T01:54:34.542 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T01:54:34.543 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T01:54:34.544 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T01:54:34.544 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T01:54:34.544 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T01:54:34.544 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T01:54:34.544 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T01:54:34.544 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T01:54:34.544 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T01:54:34.544 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T01:54:34.545 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T01:54:34.545 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T01:54:34.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T01:54:34.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T01:54:34.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T01:54:34.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T01:54:34.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T01:54:34.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T01:54:34.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T01:54:34.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T01:54:34.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T01:54:34.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T01:54:34.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T01:54:34.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T01:54:34.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T01:54:34.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T01:54:34.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T01:54:34.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T01:54:34.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T01:54:34.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T01:54:34.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T01:54:34.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T01:54:34.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T01:54:34.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T01:54:34.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T01:54:34.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T01:54:34.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T01:54:34.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T01:54:34.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T01:54:34.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T01:54:34.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T01:54:34.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T01:54:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T01:54:34.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T01:54:34.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T01:54:34.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T01:54:34.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T01:54:34.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T01:54:34.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T01:54:34.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T01:54:34.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T01:54:34.573 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T01:54:34.573 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T01:54:34.573 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T01:54:34.573 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T01:54:34.573 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T01:54:34.573 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T01:54:34.573 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T01:54:34.573 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T01:54:34.574 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T01:54:34.575 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T01:54:34.575 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T01:54:34.575 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T01:54:34.575 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T01:54:34.575 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T01:54:34.575 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T01:54:34.575 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T01:54:34.575 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T01:54:34.575 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T01:54:34.575 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T01:54:36.724 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:54:36.724 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-16T01:54:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T01:54:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T01:54:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T01:54:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T01:54:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T01:54:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T01:54:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T01:54:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T01:54:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T01:54:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T01:54:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T01:54:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T01:54:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T01:54:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T01:54:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T01:54:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T01:54:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T01:54:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T01:54:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T01:54:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T01:54:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T01:54:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T01:54:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T01:54:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T01:54:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T01:54:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T01:54:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T01:54:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T01:54:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T01:54:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T01:54:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T01:54:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T01:54:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T01:54:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T01:54:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T01:54:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T01:54:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T01:54:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T01:54:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T01:54:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T01:54:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T01:54:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T01:54:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T01:54:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T01:54:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T01:54:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T01:54:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T01:54:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:54:36.741 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-16T01:54:37.311 DEBUG:teuthology.parallel:result is None 2024-09-16T01:54:37.312 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:54:37.460 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T01:54:37.504 INFO:teuthology.orchestra.run.smithi103.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T01:54:37.505 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T01:54:37.505 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T01:54:37.508 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:54:37.648 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T01:54:37.675 INFO:teuthology.orchestra.run.smithi136.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T01:54:37.675 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T01:54:37.675 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T01:54:37.677 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:54:37.812 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T01:54:37.839 INFO:teuthology.orchestra.run.smithi172.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T01:54:37.839 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T01:54:37.839 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T01:54:37.841 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T01:54:37.841 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:54:37.842 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T01:54:37.906 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:54:37.906 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T01:54:37.939 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:54:37.939 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T01:54:37.972 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T01:54:37.972 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:54:37.972 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T01:54:38.029 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T01:54:38.130 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:54:38.130 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T01:54:38.160 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T01:54:38.230 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:54:38.230 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T01:54:38.262 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T01:54:38.329 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T01:54:38.329 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:54:38.329 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T01:54:38.386 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T01:54:38.484 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:54:38.485 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T01:54:38.510 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T01:54:38.575 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:54:38.575 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T01:54:38.601 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T01:54:38.666 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T01:54:38.667 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:54:38.667 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T01:54:38.723 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T01:54:38.820 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:54:38.820 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T01:54:38.845 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T01:54:38.913 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:54:38.913 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T01:54:38.938 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T01:54:39.004 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T01:54:39.100 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-16T01:54:39.104 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:54:39.104 INFO:tasks.cephadm:Cluster fsid is a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:54:39.105 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T01:54:39.105 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.103', 'mon.b': '172.21.15.136', 'mon.c': '172.21.15.172'} 2024-09-16T01:54:39.105 INFO:tasks.cephadm:First mon is mon.a on smithi103 2024-09-16T01:54:39.105 INFO:tasks.cephadm:First mgr is x 2024-09-16T01:54:39.105 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T01:54:39.105 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-09-16T01:54:39.162 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-09-16T01:54:39.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname $(hostname -s) 2024-09-16T01:54:39.214 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T01:54:39.214 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:54:39.355 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T01:54:39.492 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:54:39.494 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T01:54:39.494 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T01:54:39.494 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T01:54:39.919 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 01:54 /home/ubuntu/cephtest/cephadm 2024-09-16T01:54:39.919 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T01:54:40.283 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 01:54 /home/ubuntu/cephtest/cephadm 2024-09-16T01:54:40.283 DEBUG:teuthology.orchestra.run.smithi172:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T01:54:40.652 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 01:54 /home/ubuntu/cephtest/cephadm 2024-09-16T01:54:40.653 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T01:54:40.695 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T01:54:40.716 DEBUG:teuthology.orchestra.run.smithi172:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T01:54:40.751 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-16T01:54:40.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T01:54:40.755 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T01:54:40.758 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T01:54:42.343 INFO:teuthology.orchestra.run.smithi172.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:54:42.404 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:54:42.785 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:55:13.075 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-16T01:55:13.075 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T01:55:13.075 INFO:teuthology.orchestra.run.smithi172.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T01:55:13.075 INFO:teuthology.orchestra.run.smithi172.stdout: "repo_digests": [ 2024-09-16T01:55:13.075 INFO:teuthology.orchestra.run.smithi172.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T01:55:13.075 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-16T01:55:13.076 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-16T01:55:13.464 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T01:55:13.464 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T01:55:13.464 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T01:55:13.464 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-09-16T01:55:13.464 INFO:teuthology.orchestra.run.smithi136.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T01:55:13.464 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T01:55:13.464 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T01:55:15.916 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T01:55:15.917 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T01:55:15.917 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T01:55:15.917 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-09-16T01:55:15.917 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T01:55:15.917 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-16T01:55:15.917 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T01:55:15.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-09-16T01:55:16.020 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-09-16T01:55:16.053 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph 2024-09-16T01:55:16.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-09-16T01:55:16.143 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-09-16T01:55:16.173 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /etc/ceph 2024-09-16T01:55:16.203 INFO:tasks.cephadm:Writing seed config... 2024-09-16T01:55:16.204 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-16T01:55:16.204 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-16T01:55:16.204 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-16T01:55:16.204 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-16T01:55:16.204 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-16T01:55:16.204 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-16T01:55:16.204 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-16T01:55:16.204 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-16T01:55:16.204 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-16T01:55:16.205 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-16T01:55:16.206 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T01:55:16.207 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-16T01:55:16.208 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:55:16.208 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T01:55:16.244 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a20dd7a4-73ce-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-16T01:55:16.244 DEBUG:teuthology.orchestra.run.smithi103:mon.a> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T01:55:16.289 DEBUG:teuthology.orchestra.run.smithi103:mgr.x> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T01:55:16.333 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T01:55:16.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.103 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T01:55:16.593 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:55:16.593 INFO:teuthology.orchestra.run.smithi103.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', 'a20dd7a4-73ce-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.103', '--skip-admin-label'] 2024-09-16T01:55:16.594 INFO:teuthology.orchestra.run.smithi103.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T01:55:16.594 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying podman|docker is present... 2024-09-16T01:55:16.639 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.2.2 2024-09-16T01:55:16.640 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying lvm2 is present... 2024-09-16T01:55:16.640 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying time synchronization is in place... 2024-09-16T01:55:16.653 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T01:55:16.653 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T01:55:16.665 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T01:55:16.666 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-16T01:55:16.679 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-09-16T01:55:16.691 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-09-16T01:55:16.691 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-09-16T01:55:16.691 INFO:teuthology.orchestra.run.smithi103.stdout:Repeating the final host check... 2024-09-16T01:55:16.748 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.2.2 2024-09-16T01:55:16.748 INFO:teuthology.orchestra.run.smithi103.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T01:55:16.748 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl is present 2024-09-16T01:55:16.748 INFO:teuthology.orchestra.run.smithi103.stdout:lvcreate is present 2024-09-16T01:55:16.760 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T01:55:16.761 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T01:55:16.773 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T01:55:16.773 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-16T01:55:16.786 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-09-16T01:55:16.799 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-09-16T01:55:16.799 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-09-16T01:55:16.799 INFO:teuthology.orchestra.run.smithi103.stdout:Host looks OK 2024-09-16T01:55:16.799 INFO:teuthology.orchestra.run.smithi103.stdout:Cluster fsid: a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:16.799 INFO:teuthology.orchestra.run.smithi103.stdout:Acquiring lock 140642796856320 on /run/cephadm/a20dd7a4-73ce-11ef-bceb-c7b262605968.lock 2024-09-16T01:55:16.799 INFO:teuthology.orchestra.run.smithi103.stdout:Lock 140642796856320 acquired on /run/cephadm/a20dd7a4-73ce-11ef-bceb-c7b262605968.lock 2024-09-16T01:55:16.800 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 3300 ... 2024-09-16T01:55:16.800 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 6789 ... 2024-09-16T01:55:16.800 INFO:teuthology.orchestra.run.smithi103.stdout:Base mon IP(s) is [172.21.15.103:3300, 172.21.15.103:6789], mon addrv is [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-16T01:55:16.807 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.103 metric 100 2024-09-16T01:55:16.807 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.103 metric 100 2024-09-16T01:55:16.813 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T01:55:16.813 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-16T01:55:16.813 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-16T01:55:16.819 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T01:55:16.819 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T01:55:16.819 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T01:55:16.819 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-16T01:55:16.819 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-16T01:55:16.819 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T01:55:16.820 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-09-16T01:55:16.820 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-09-16T01:55:16.820 INFO:teuthology.orchestra.run.smithi103.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T01:55:16.820 INFO:teuthology.orchestra.run.smithi103.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T01:55:16.821 INFO:teuthology.orchestra.run.smithi103.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:55:17.350 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T01:55:17.351 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:55:17.351 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T01:55:17.351 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-16T01:55:17.351 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T01:55:17.351 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T01:55:17.351 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T01:55:18.355 INFO:teuthology.orchestra.run.smithi103.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T01:55:18.355 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T01:55:18.355 INFO:teuthology.orchestra.run.smithi103.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T01:55:19.360 INFO:teuthology.orchestra.run.smithi103.stdout:stat: stdout 167 167 2024-09-16T01:55:19.360 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial keys... 2024-09-16T01:55:30.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQAMkOdmNyUPDxAA+XyJRdtey2fdd7KBDHg96Q== 2024-09-16T01:55:31.193 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQASkOdmTMl3IBAAavPBVIUO/dxaOjxoOjgm8A== 2024-09-16T01:55:32.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQATkOdma4k+IhAA2aE2Dnk2048CMXH5baP2Yg== 2024-09-16T01:55:32.308 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial monmap... 2024-09-16T01:55:33.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T01:55:33.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-16T01:55:33.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:33.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T01:55:33.548 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool for a [v2:172.21.15.103:3300,v1:172.21.15.103:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T01:55:33.548 INFO:teuthology.orchestra.run.smithi103.stdout:setting min_mon_release = quincy 2024-09-16T01:55:33.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: set fsid to a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:33.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T01:55:33.548 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:55:33.549 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mon... 2024-09-16T01:55:35.646 INFO:teuthology.orchestra.run.smithi103.stdout:create mon.a on 2024-09-16T01:55:35.944 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T01:55:36.161 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T01:55:36.415 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968.target. 2024-09-16T01:55:36.415 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968.target. 2024-09-16T01:55:36.757 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.a 2024-09-16T01:55:36.757 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T01:55:37.038 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968.target.wants/ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@.service. 2024-09-16T01:55:37.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:37 smithi103 podman[30015]: 2024-09-16 01:55:37.608090522 +0000 UTC m=+0.339949864 container start 70cf838123c000221905b3e77a4dbcf807a413354ad5f407de6a172d172e92fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T01:55:37.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:37 smithi103 ceph-mon[30034]: mkfs a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:37.738 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:55:37.738 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-16T01:55:37.751 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:55:37.751 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-16T01:55:37.751 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-16T01:55:37.751 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T01:55:37.752 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon to start... 2024-09-16T01:55:37.752 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon... 2024-09-16T01:55:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:37 smithi103 ceph-mon[30034]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T01:55:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:37 smithi103 bash[30015]: 70cf838123c000221905b3e77a4dbcf807a413354ad5f407de6a172d172e92fa 2024-09-16T01:55:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:37 smithi103 systemd[1]: Started Ceph mon.a for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:55:38.975 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T01:55:38.975 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: monmap epoch 1 2024-09-16T01:55:38.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:38.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: last_changed 2024-09-16T01:55:32.719520+0000 2024-09-16T01:55:38.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:55:38.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: min_mon_release 19 (squid) 2024-09-16T01:55:38.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: election_strategy: 1 2024-09-16T01:55:38.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:55:38.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: fsmap 2024-09-16T01:55:38.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T01:55:38.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: mgrmap e1: no daemons active 2024-09-16T01:55:38.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:38 smithi103 ceph-mon[30034]: from='client.? 172.21.15.103:0/1596682763' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T01:55:38.986 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T01:55:38.986 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout id: a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout services: 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.660126s) 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout data: 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T01:55:38.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.250+0000 7f4567e00640 1 Processor -- start 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.251+0000 7f4567e00640 1 -- start start 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.251+0000 7f4567e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f4568107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.251+0000 7f4567e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45681080a0 con 0x7f45681076d0 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.252+0000 7f4566a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f4568107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.252+0000 7f4566a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f4568107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35766/0 (socket says 172.21.15.103:35766) 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.252+0000 7f4566a00640 1 -- 172.21.15.103:0/2382764374 learned_addr learned my addr 172.21.15.103:0/2382764374 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.252+0000 7f4566a00640 1 -- 172.21.15.103:0/2382764374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4568108830 con 0x7f45681076d0 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.255+0000 7f4566a00640 1 --2- 172.21.15.103:0/2382764374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f4568107ad0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4558009820 tx=0x7f455802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b5a0d739cfce2c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:38.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.255+0000 7f4566000640 1 -- 172.21.15.103:0/2382764374 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558037070 con 0x7f45681076d0 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.255+0000 7f4566000640 1 -- 172.21.15.103:0/2382764374 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4558032440 con 0x7f45681076d0 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.255+0000 7f4566000640 1 -- 172.21.15.103:0/2382764374 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558032740 con 0x7f45681076d0 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.256+0000 7f4567e00640 1 -- 172.21.15.103:0/2382764374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 msgr2=0x7f4568107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.256+0000 7f4567e00640 1 --2- 172.21.15.103:0/2382764374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f4568107ad0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4558009820 tx=0x7f455802a270 comp rx=0 tx=0).stop 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.256+0000 7f4567e00640 1 -- 172.21.15.103:0/2382764374 shutdown_connections 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.256+0000 7f4567e00640 1 --2- 172.21.15.103:0/2382764374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f4568107ad0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.256+0000 7f4567e00640 1 -- 172.21.15.103:0/2382764374 >> 172.21.15.103:0/2382764374 conn(0x7f4568102ea0 msgr2=0x7f45681052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.256+0000 7f4567e00640 1 -- 172.21.15.103:0/2382764374 shutdown_connections 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.256+0000 7f4567e00640 1 -- 172.21.15.103:0/2382764374 wait complete. 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.257+0000 7f4567e00640 1 Processor -- start 2024-09-16T01:55:38.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.257+0000 7f4567e00640 1 -- start start 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.257+0000 7f4567e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f456807db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.257+0000 7f4567e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4568108dd0 con 0x7f45681076d0 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.258+0000 7f4566a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f456807db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.258+0000 7f4566a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f456807db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35772/0 (socket says 172.21.15.103:35772) 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.258+0000 7f4566a00640 1 -- 172.21.15.103:0/1596682763 learned_addr learned my addr 172.21.15.103:0/1596682763 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.258+0000 7f4566a00640 1 -- 172.21.15.103:0/1596682763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456807e060 con 0x7f45681076d0 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.258+0000 7f4566a00640 1 --2- 172.21.15.103:0/1596682763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f456807db20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f45580090b0 tx=0x7f45580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.259+0000 7f455f400640 1 -- 172.21.15.103:0/1596682763 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558041070 con 0x7f45681076d0 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.259+0000 7f455f400640 1 -- 172.21.15.103:0/1596682763 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455803c7a0 con 0x7f45681076d0 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.259+0000 7f455f400640 1 -- 172.21.15.103:0/1596682763 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558037040 con 0x7f45681076d0 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.259+0000 7f4567e00640 1 -- 172.21.15.103:0/1596682763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456807c250 con 0x7f45681076d0 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.259+0000 7f4567e00640 1 -- 172.21.15.103:0/1596682763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456807c790 con 0x7f45681076d0 2024-09-16T01:55:38.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.259+0000 7f455f400640 1 -- 172.21.15.103:0/1596682763 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4558044070 con 0x7f45681076d0 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.259+0000 7f455f400640 1 -- 172.21.15.103:0/1596682763 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f455803d3f0 con 0x7f45681076d0 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.260+0000 7f4567e00640 1 -- 172.21.15.103:0/1596682763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4568109420 con 0x7f45681076d0 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.263+0000 7f455f400640 1 -- 172.21.15.103:0/1596682763 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4558046450 con 0x7f45681076d0 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.324+0000 7f4567e00640 1 -- 172.21.15.103:0/1596682763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f456807ce90 con 0x7f45681076d0 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.324+0000 7f455f400640 1 -- 172.21.15.103:0/1596682763 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f4558046630 con 0x7f45681076d0 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.325+0000 7f4567e00640 1 -- 172.21.15.103:0/1596682763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 msgr2=0x7f456807db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.325+0000 7f4567e00640 1 --2- 172.21.15.103:0/1596682763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f456807db20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f45580090b0 tx=0x7f45580046c0 comp rx=0 tx=0).stop 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.325+0000 7f4567e00640 1 -- 172.21.15.103:0/1596682763 shutdown_connections 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.325+0000 7f4567e00640 1 --2- 172.21.15.103:0/1596682763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45681076d0 0x7f456807db20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.325+0000 7f4567e00640 1 -- 172.21.15.103:0/1596682763 >> 172.21.15.103:0/1596682763 conn(0x7f4568102ea0 msgr2=0x7f456818f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.326+0000 7f4567e00640 1 -- 172.21.15.103:0/1596682763 shutdown_connections 2024-09-16T01:55:38.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:38.326+0000 7f4567e00640 1 -- 172.21.15.103:0/1596682763 wait complete. 2024-09-16T01:55:38.992 INFO:teuthology.orchestra.run.smithi103.stdout:mon is available 2024-09-16T01:55:38.992 INFO:teuthology.orchestra.run.smithi103.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T01:55:39.870 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:39 smithi103 ceph-mon[30034]: from='client.? 172.21.15.103:0/2272465222' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T01:55:39.870 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:39 smithi103 ceph-mon[30034]: from='client.? 172.21.15.103:0/2272465222' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T01:55:40.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:40.286 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T01:55:40.286 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:40.286 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-16T01:55:40.286 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T01:55:40.286 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T01:55:40.286 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T01:55:40.286 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T01:55:40.286 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:40.286 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T01:55:40.286 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T01:55:40.287 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:40.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T01:55:40.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T01:55:40.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T01:55:40.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T01:55:40.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T01:55:40.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T01:55:40.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.456+0000 7f797d400640 1 Processor -- start 2024-09-16T01:55:40.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.456+0000 7f797d400640 1 -- start start 2024-09-16T01:55:40.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.456+0000 7f797d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f7978109e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.456+0000 7f797d400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f797810a3e0 con 0x7f7978109a10 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.457+0000 7f7976800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f7978109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.457+0000 7f7976800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f7978109e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35788/0 (socket says 172.21.15.103:35788) 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.457+0000 7f7976800640 1 -- 172.21.15.103:0/2460058790 learned_addr learned my addr 172.21.15.103:0/2460058790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.457+0000 7f7976800640 1 -- 172.21.15.103:0/2460058790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797810abc0 con 0x7f7978109a10 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.458+0000 7f7976800640 1 --2- 172.21.15.103:0/2460058790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f7978109e10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7968009820 tx=0x7f796802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b37751b3ef6509d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.458+0000 7f7975400640 1 -- 172.21.15.103:0/2460058790 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7968037070 con 0x7f7978109a10 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.458+0000 7f7975400640 1 -- 172.21.15.103:0/2460058790 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7968032440 con 0x7f7978109a10 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.458+0000 7f7975400640 1 -- 172.21.15.103:0/2460058790 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7968032740 con 0x7f7978109a10 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.459+0000 7f797d400640 1 -- 172.21.15.103:0/2460058790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 msgr2=0x7f7978109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.459+0000 7f797d400640 1 --2- 172.21.15.103:0/2460058790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f7978109e10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7968009820 tx=0x7f796802a270 comp rx=0 tx=0).stop 2024-09-16T01:55:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.459+0000 7f797d400640 1 -- 172.21.15.103:0/2460058790 shutdown_connections 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.459+0000 7f797d400640 1 --2- 172.21.15.103:0/2460058790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f7978109e10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.459+0000 7f797d400640 1 -- 172.21.15.103:0/2460058790 >> 172.21.15.103:0/2460058790 conn(0x7f797807d030 msgr2=0x7f797807d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.459+0000 7f797d400640 1 -- 172.21.15.103:0/2460058790 shutdown_connections 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.459+0000 7f797d400640 1 -- 172.21.15.103:0/2460058790 wait complete. 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.460+0000 7f797d400640 1 Processor -- start 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.460+0000 7f797d400640 1 -- start start 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.461+0000 7f797d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f797819d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.461+0000 7f797d400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f797810b160 con 0x7f7978109a10 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.461+0000 7f7976800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f797819d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.461+0000 7f7976800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f797819d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35800/0 (socket says 172.21.15.103:35800) 2024-09-16T01:55:40.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.461+0000 7f7976800640 1 -- 172.21.15.103:0/2272465222 learned_addr learned my addr 172.21.15.103:0/2272465222 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.461+0000 7f7976800640 1 -- 172.21.15.103:0/2272465222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797819d6d0 con 0x7f7978109a10 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.461+0000 7f7976800640 1 --2- 172.21.15.103:0/2272465222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f797819d190 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f79680090b0 tx=0x7f79680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.462+0000 7f796f400640 1 -- 172.21.15.103:0/2272465222 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7968041070 con 0x7f7978109a10 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.462+0000 7f796f400640 1 -- 172.21.15.103:0/2272465222 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796803c7a0 con 0x7f7978109a10 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.462+0000 7f797d400640 1 -- 172.21.15.103:0/2272465222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797819d960 con 0x7f7978109a10 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.462+0000 7f796f400640 1 -- 172.21.15.103:0/2272465222 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7968037040 con 0x7f7978109a10 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.462+0000 7f797d400640 1 -- 172.21.15.103:0/2272465222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79781a0650 con 0x7f7978109a10 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.462+0000 7f796f400640 1 -- 172.21.15.103:0/2272465222 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7968044070 con 0x7f7978109a10 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.462+0000 7f796f400640 1 -- 172.21.15.103:0/2272465222 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f796803d3f0 con 0x7f7978109a10 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.463+0000 7f797d400640 1 -- 172.21.15.103:0/2272465222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797810b7b0 con 0x7f7978109a10 2024-09-16T01:55:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.466+0000 7f796f400640 1 -- 172.21.15.103:0/2272465222 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7968046450 con 0x7f7978109a10 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.526+0000 7f797d400640 1 -- 172.21.15.103:0/2272465222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f79781a1d20 con 0x7f7978109a10 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.530+0000 7f796f400640 1 -- 172.21.15.103:0/2272465222 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f7968046630 con 0x7f7978109a10 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.531+0000 7f796f400640 1 -- 172.21.15.103:0/2272465222 <== mon.0 v2:172.21.15.103:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7968056070 con 0x7f7978109a10 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.531+0000 7f797d400640 1 -- 172.21.15.103:0/2272465222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 msgr2=0x7f797819d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.531+0000 7f797d400640 1 --2- 172.21.15.103:0/2272465222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f797819d190 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f79680090b0 tx=0x7f79680046c0 comp rx=0 tx=0).stop 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.532+0000 7f797d400640 1 -- 172.21.15.103:0/2272465222 shutdown_connections 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.532+0000 7f797d400640 1 --2- 172.21.15.103:0/2272465222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7978109a10 0x7f797819d190 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.532+0000 7f797d400640 1 -- 172.21.15.103:0/2272465222 >> 172.21.15.103:0/2272465222 conn(0x7f797807d030 msgr2=0x7f7978191480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.532+0000 7f797d400640 1 -- 172.21.15.103:0/2272465222 shutdown_connections 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:39.532+0000 7f797d400640 1 -- 172.21.15.103:0/2272465222 wait complete. 2024-09-16T01:55:40.292 INFO:teuthology.orchestra.run.smithi103.stdout:Generating new minimal ceph.conf... 2024-09-16T01:55:41.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:40 smithi103 ceph-mon[30034]: from='client.? 172.21.15.103:0/3035498283' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:55:41.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.730+0000 7fc3a4a00640 1 Processor -- start 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.730+0000 7fc3a4a00640 1 -- start start 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.730+0000 7fc3a4a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a0107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.730+0000 7fc3a4a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a0108000 con 0x7fc3a0107630 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.731+0000 7fc39e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a0107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.731+0000 7fc39e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a0107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35810/0 (socket says 172.21.15.103:35810) 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.731+0000 7fc39e800640 1 -- 172.21.15.103:0/315636980 learned_addr learned my addr 172.21.15.103:0/315636980 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.731+0000 7fc39e800640 1 -- 172.21.15.103:0/315636980 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a0108830 con 0x7fc3a0107630 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.732+0000 7fc39e800640 1 --2- 172.21.15.103:0/315636980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a0107a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc390009820 tx=0x7fc39002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f7e4f9a7d633a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.732+0000 7fc39de00640 1 -- 172.21.15.103:0/315636980 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc390037070 con 0x7fc3a0107630 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.732+0000 7fc39de00640 1 -- 172.21.15.103:0/315636980 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc390032900 con 0x7fc3a0107630 2024-09-16T01:55:41.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.732+0000 7fc39de00640 1 -- 172.21.15.103:0/315636980 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc390030850 con 0x7fc3a0107630 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.733+0000 7fc3a4a00640 1 -- 172.21.15.103:0/315636980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 msgr2=0x7fc3a0107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.733+0000 7fc3a4a00640 1 --2- 172.21.15.103:0/315636980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a0107a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc390009820 tx=0x7fc39002a270 comp rx=0 tx=0).stop 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.733+0000 7fc3a4a00640 1 -- 172.21.15.103:0/315636980 shutdown_connections 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.733+0000 7fc3a4a00640 1 --2- 172.21.15.103:0/315636980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a0107a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.733+0000 7fc3a4a00640 1 -- 172.21.15.103:0/315636980 >> 172.21.15.103:0/315636980 conn(0x7fc3a0102dc0 msgr2=0x7fc3a0105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.733+0000 7fc3a4a00640 1 -- 172.21.15.103:0/315636980 shutdown_connections 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.733+0000 7fc3a4a00640 1 -- 172.21.15.103:0/315636980 wait complete. 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.734+0000 7fc3a4a00640 1 Processor -- start 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.734+0000 7fc3a4a00640 1 -- start start 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.734+0000 7fc3a4a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a007db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.734+0000 7fc3a4a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a010bbe0 con 0x7fc3a0107630 2024-09-16T01:55:41.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.735+0000 7fc39e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a007db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.735+0000 7fc39e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a007db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35824/0 (socket says 172.21.15.103:35824) 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.735+0000 7fc39e800640 1 -- 172.21.15.103:0/3035498283 learned_addr learned my addr 172.21.15.103:0/3035498283 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.735+0000 7fc39e800640 1 -- 172.21.15.103:0/3035498283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a007e060 con 0x7fc3a0107630 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.735+0000 7fc39e800640 1 --2- 172.21.15.103:0/3035498283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a007db20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3900097f0 tx=0x7fc390030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.735+0000 7fc397400640 1 -- 172.21.15.103:0/3035498283 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc390037040 con 0x7fc3a0107630 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.735+0000 7fc397400640 1 -- 172.21.15.103:0/3035498283 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc390040070 con 0x7fc3a0107630 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.735+0000 7fc3a4a00640 1 -- 172.21.15.103:0/3035498283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a007c250 con 0x7fc3a0107630 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.736+0000 7fc397400640 1 -- 172.21.15.103:0/3035498283 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc39003a390 con 0x7fc3a0107630 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.736+0000 7fc3a4a00640 1 -- 172.21.15.103:0/3035498283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a007c730 con 0x7fc3a0107630 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.736+0000 7fc397400640 1 -- 172.21.15.103:0/3035498283 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc39003a5e0 con 0x7fc3a0107630 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.736+0000 7fc397400640 1 -- 172.21.15.103:0/3035498283 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc390045950 con 0x7fc3a0107630 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.737+0000 7fc3a4a00640 1 -- 172.21.15.103:0/3035498283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a0107ab0 con 0x7fc3a0107630 2024-09-16T01:55:41.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.739+0000 7fc397400640 1 -- 172.21.15.103:0/3035498283 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc390045460 con 0x7fc3a0107630 2024-09-16T01:55:41.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.799+0000 7fc3a4a00640 1 -- 172.21.15.103:0/3035498283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc3a007cc60 con 0x7fc3a0107630 2024-09-16T01:55:41.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.799+0000 7fc397400640 1 -- 172.21.15.103:0/3035498283 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fc39003b430 con 0x7fc3a0107630 2024-09-16T01:55:41.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.801+0000 7fc3a4a00640 1 -- 172.21.15.103:0/3035498283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 msgr2=0x7fc3a007db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:41.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.801+0000 7fc3a4a00640 1 --2- 172.21.15.103:0/3035498283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a007db20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3900097f0 tx=0x7fc390030d80 comp rx=0 tx=0).stop 2024-09-16T01:55:41.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.801+0000 7fc3a4a00640 1 -- 172.21.15.103:0/3035498283 shutdown_connections 2024-09-16T01:55:41.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.801+0000 7fc3a4a00640 1 --2- 172.21.15.103:0/3035498283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3a0107630 0x7fc3a007db20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:41.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.801+0000 7fc3a4a00640 1 -- 172.21.15.103:0/3035498283 >> 172.21.15.103:0/3035498283 conn(0x7fc3a0102dc0 msgr2=0x7fc3a0103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:41.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.801+0000 7fc3a4a00640 1 -- 172.21.15.103:0/3035498283 shutdown_connections 2024-09-16T01:55:41.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:40.801+0000 7fc3a4a00640 1 -- 172.21.15.103:0/3035498283 wait complete. 2024-09-16T01:55:41.451 INFO:teuthology.orchestra.run.smithi103.stdout:Restarting the monitor... 2024-09-16T01:55:41.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:41 smithi103 systemd[1]: Stopping Ceph mon.a for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:55:42.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:41 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a[30030]: 2024-09-16T01:55:41.809+0000 7f337fa00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T01:55:42.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:41 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a[30030]: 2024-09-16T01:55:41.809+0000 7f337fa00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T01:55:42.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:41 smithi103 podman[30277]: 2024-09-16 01:55:41.984311668 +0000 UTC m=+0.306580330 container died 70cf838123c000221905b3e77a4dbcf807a413354ad5f407de6a172d172e92fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T01:55:42.624 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:42 smithi103 podman[30277]: 2024-09-16 01:55:42.397380297 +0000 UTC m=+0.719648960 container cleanup 70cf838123c000221905b3e77a4dbcf807a413354ad5f407de6a172d172e92fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-8921c8e) 2024-09-16T01:55:42.624 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:42 smithi103 bash[30277]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a 2024-09-16T01:55:42.876 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:42 smithi103 podman[30291]: 2024-09-16 01:55:42.623463041 +0000 UTC m=+0.634645550 container remove 70cf838123c000221905b3e77a4dbcf807a413354ad5f407de6a172d172e92fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T01:55:42.877 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:42 smithi103 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T01:55:42.877 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:42 smithi103 systemd[1]: Stopped Ceph mon.a for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:55:42.877 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:42 smithi103 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.a.service: Consumed 1.205s CPU time. 2024-09-16T01:55:42.877 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:42 smithi103 systemd[1]: Starting Ceph mon.a for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:55:43.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 podman[30379]: 2024-09-16 01:55:43.182668922 +0000 UTC m=+0.119998692 container create e60b64d004a06b198ed5ddbd90e8194edbffd30db141086ff1f9287511db2fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T01:55:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 podman[30379]: 2024-09-16 01:55:43.090074979 +0000 UTC m=+0.027404750 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:55:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 podman[30379]: 2024-09-16 01:55:43.359882534 +0000 UTC m=+0.297212305 container init e60b64d004a06b198ed5ddbd90e8194edbffd30db141086ff1f9287511db2fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T01:55:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 podman[30379]: 2024-09-16 01:55:43.366811551 +0000 UTC m=+0.304141322 container start e60b64d004a06b198ed5ddbd90e8194edbffd30db141086ff1f9287511db2fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T01:55:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T01:55:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T01:55:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: pidfile_write: ignore empty --pid-file 2024-09-16T01:55:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: load: jerasure load: lrc 2024-09-16T01:55:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: RocksDB version: 7.9.2 2024-09-16T01:55:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Git sha 0 2024-09-16T01:55:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T01:55:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: DB SUMMARY 2024-09-16T01:55:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: DB Session ID: MJRLF8EPKP9NL9RRE1ZX 2024-09-16T01:55:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: CURRENT file: CURRENT 2024-09-16T01:55:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T01:55:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T01:55:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T01:55:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106812 ; 2024-09-16T01:55:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.error_if_exists: 0 2024-09-16T01:55:43.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.create_if_missing: 0 2024-09-16T01:55:43.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.paranoid_checks: 1 2024-09-16T01:55:43.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T01:55:43.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T01:55:43.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T01:55:43.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.env: 0x55d8047fc940 2024-09-16T01:55:43.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T01:55:43.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.info_log: 0x55d8349c36a0 2024-09-16T01:55:43.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T01:55:43.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.statistics: (nil) 2024-09-16T01:55:43.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.use_fsync: 0 2024-09-16T01:55:43.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_log_file_size: 0 2024-09-16T01:55:43.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T01:55:43.478 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T01:55:43.478 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T01:55:43.478 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T01:55:43.478 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.allow_fallocate: 1 2024-09-16T01:55:43.478 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T01:55:43.478 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T01:55:43.478 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.use_direct_reads: 0 2024-09-16T01:55:43.480 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T01:55:43.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T01:55:43.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.db_log_dir: 2024-09-16T01:55:43.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.wal_dir: 2024-09-16T01:55:43.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T01:55:43.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T01:55:43.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T01:55:43.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T01:55:43.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T01:55:43.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.write_buffer_manager: 0x55d8349d2dc0 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T01:55:43.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T01:55:43.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.unordered_write: 0 2024-09-16T01:55:43.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T01:55:43.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T01:55:43.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T01:55:43.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T01:55:43.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.row_cache: None 2024-09-16T01:55:43.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.wal_filter: None 2024-09-16T01:55:43.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T01:55:43.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T01:55:43.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.two_write_queues: 0 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.wal_compression: 0 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.atomic_flush: 0 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.log_readahead_size: 0 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T01:55:43.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_background_jobs: 2 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_background_compactions: -1 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_subcompactions: 1 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T01:55:43.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_open_files: -1 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_background_flushes: -1 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Compression algorithms supported: 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: kZSTD supported: 0 2024-09-16T01:55:43.486 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: kXpressCompression supported: 0 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: kBZip2Compression supported: 0 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: kLZ4Compression supported: 1 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: kZlibCompression supported: 1 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: kSnappyCompression supported: 1 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T01:55:43.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T01:55:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.merge_operator: 2024-09-16T01:55:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_filter: None 2024-09-16T01:55:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T01:55:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T01:55:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T01:55:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T01:55:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d8349c3300) 2024-09-16T01:55:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks: 1 2024-09-16T01:55:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T01:55:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T01:55:43.489 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_top_level_index_and_filter: 1 2024-09-16T01:55:43.489 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_type: 0 2024-09-16T01:55:43.489 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_index_type: 0 2024-09-16T01:55:43.489 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_shortening: 1 2024-09-16T01:55:43.489 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T01:55:43.489 INFO:journalctl@ceph.mon.a.smithi103.stdout: checksum: 4 2024-09-16T01:55:43.489 INFO:journalctl@ceph.mon.a.smithi103.stdout: no_block_cache: 0 2024-09-16T01:55:43.489 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache: 0x55d8349b71f0 2024-09-16T01:55:43.489 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_name: BinnedLRUCache 2024-09-16T01:55:43.489 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_options: 2024-09-16T01:55:43.490 INFO:journalctl@ceph.mon.a.smithi103.stdout: capacity : 536870912 2024-09-16T01:55:43.490 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_shard_bits : 4 2024-09-16T01:55:43.490 INFO:journalctl@ceph.mon.a.smithi103.stdout: strict_capacity_limit : 0 2024-09-16T01:55:43.490 INFO:journalctl@ceph.mon.a.smithi103.stdout: high_pri_pool_ratio: 0.000 2024-09-16T01:55:43.490 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_compressed: (nil) 2024-09-16T01:55:43.490 INFO:journalctl@ceph.mon.a.smithi103.stdout: persistent_cache: (nil) 2024-09-16T01:55:43.490 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size: 4096 2024-09-16T01:55:43.490 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size_deviation: 10 2024-09-16T01:55:43.490 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_restart_interval: 16 2024-09-16T01:55:43.490 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_block_restart_interval: 1 2024-09-16T01:55:43.492 INFO:journalctl@ceph.mon.a.smithi103.stdout: metadata_block_size: 4096 2024-09-16T01:55:43.492 INFO:journalctl@ceph.mon.a.smithi103.stdout: partition_filters: 0 2024-09-16T01:55:43.492 INFO:journalctl@ceph.mon.a.smithi103.stdout: use_delta_encoding: 1 2024-09-16T01:55:43.492 INFO:journalctl@ceph.mon.a.smithi103.stdout: filter_policy: bloomfilter 2024-09-16T01:55:43.492 INFO:journalctl@ceph.mon.a.smithi103.stdout: whole_key_filtering: 1 2024-09-16T01:55:43.492 INFO:journalctl@ceph.mon.a.smithi103.stdout: verify_compression: 0 2024-09-16T01:55:43.492 INFO:journalctl@ceph.mon.a.smithi103.stdout: read_amp_bytes_per_bit: 0 2024-09-16T01:55:43.492 INFO:journalctl@ceph.mon.a.smithi103.stdout: format_version: 5 2024-09-16T01:55:43.492 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_index_compression: 1 2024-09-16T01:55:43.492 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_align: 0 2024-09-16T01:55:43.493 INFO:journalctl@ceph.mon.a.smithi103.stdout: max_auto_readahead_size: 262144 2024-09-16T01:55:43.493 INFO:journalctl@ceph.mon.a.smithi103.stdout: prepopulate_block_cache: 0 2024-09-16T01:55:43.493 INFO:journalctl@ceph.mon.a.smithi103.stdout: initial_auto_readahead_size: 8192 2024-09-16T01:55:43.493 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T01:55:43.493 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T01:55:43.493 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T01:55:43.493 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compression: NoCompression 2024-09-16T01:55:43.493 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T01:55:43.493 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T01:55:43.493 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T01:55:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.num_levels: 7 2024-09-16T01:55:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T01:55:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T01:55:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T01:55:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T01:55:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T01:55:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T01:55:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T01:55:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:55:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T01:55:43.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T01:55:43.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:55:43.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:55:43.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T01:55:43.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T01:55:43.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T01:55:43.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T01:55:43.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:55:43.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:55:43.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T01:55:43.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T01:55:43.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:55:43.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T01:55:43.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T01:55:43.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T01:55:43.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T01:55:43.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T01:55:43.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T01:55:43.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T01:55:43.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T01:55:43.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T01:55:43.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T01:55:43.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T01:55:43.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T01:55:43.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T01:55:43.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T01:55:43.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T01:55:43.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T01:55:43.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T01:55:43.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T01:55:43.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T01:55:43.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T01:55:43.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T01:55:43.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T01:55:43.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T01:55:43.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.inplace_update_support: 0 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.bloom_locality: 0 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.max_successive_merges: 0 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T01:55:43.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.ttl: 2592000 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.enable_blob_files: false 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.min_blob_size: 0 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T01:55:43.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 47f1f283-c65c-4306-9ba1-3b5dd51f599c 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451743410826, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451743413595, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101746, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 99631, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14671, "raw_average_key_size": 50, "raw_value_size": 91686, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 293, "num_filter_entries": 293, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726451743, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "47f1f283-c65c-4306-9ba1-3b5dd51f599c", "db_session_id": "MJRLF8EPKP9NL9RRE1ZX", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451743413885, "job": 1, "event": "recovery_finished"} 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T01:55:43.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d8349e6e00 2024-09-16T01:55:43.503 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: DB pointer 0x55d834ae2000 2024-09-16T01:55:43.503 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T01:55:43.503 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T01:55:43.503 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** DB Stats ** 2024-09-16T01:55:43.503 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:55:43.503 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T01:55:43.503 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:55:43.503 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:55:43.503 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T01:55:43.503 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: L0 2/0 101.22 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: Sum 2/0 101.22 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-16T01:55:43.504 INFO:journalctl@ceph.mon.a.smithi103.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:55:43.505 INFO:journalctl@ceph.mon.a.smithi103.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T01:55:43.505 INFO:journalctl@ceph.mon.a.smithi103.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 57.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-16T01:55:43.505 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-16T01:55:43.505 INFO:journalctl@ceph.mon.a.smithi103.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T01:55:43.505 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-16T01:55:43.505 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:55:43.505 INFO:journalctl@ceph.mon.a.smithi103.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T01:55:43.505 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T01:55:43.505 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T01:55:43.505 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative compaction: 0.00 GB write, 10.92 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval compaction: 0.00 GB write, 10.92 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache BinnedLRUCache@0x55d8349b71f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: starting mon.a rank 0 at public addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] at bind addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: mon.a@-1(???) e1 preinit fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:43.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: mon.a@-1(???).mds e1 new map 2024-09-16T01:55:43.507 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: mon.a@-1(???).mds e1 print_map 2024-09-16T01:55:43.507 INFO:journalctl@ceph.mon.a.smithi103.stdout: e1 2024-09-16T01:55:43.507 INFO:journalctl@ceph.mon.a.smithi103.stdout: btime 2024-09-16T01:55:37:666788+0000 2024-09-16T01:55:43.507 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T01:55:43.507 INFO:journalctl@ceph.mon.a.smithi103.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T01:55:43.507 INFO:journalctl@ceph.mon.a.smithi103.stdout: legacy client fscid: -1 2024-09-16T01:55:43.507 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-16T01:55:43.507 INFO:journalctl@ceph.mon.a.smithi103.stdout: No filesystems configured 2024-09-16T01:55:43.507 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T01:55:43.507 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:55:43.508 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:55:43.508 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:55:43.508 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T01:55:43.508 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 bash[30379]: e60b64d004a06b198ed5ddbd90e8194edbffd30db141086ff1f9287511db2fa1 2024-09-16T01:55:43.508 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 systemd[1]: Started Ceph mon.a for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:55:43.508 INFO:teuthology.orchestra.run.smithi103.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T01:55:43.744 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T01:55:43.744 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: monmap epoch 1 2024-09-16T01:55:43.744 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:43.744 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: last_changed 2024-09-16T01:55:32.719520+0000 2024-09-16T01:55:43.745 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:55:43.745 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: min_mon_release 19 (squid) 2024-09-16T01:55:43.745 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: election_strategy: 1 2024-09-16T01:55:43.745 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:55:43.745 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: fsmap 2024-09-16T01:55:43.745 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T01:55:43.745 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:43 smithi103 ceph-mon[30398]: mgrmap e1: no daemons active 2024-09-16T01:55:44.620 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.954+0000 7fd8ea400640 1 Processor -- start 2024-09-16T01:55:44.620 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.955+0000 7fd8ea400640 1 -- start start 2024-09-16T01:55:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.955+0000 7fd8ea400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e407c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.955+0000 7fd8ea400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e407c7b0 con 0x7fd8e407de10 2024-09-16T01:55:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.955+0000 7fd8e3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e407c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.955+0000 7fd8e3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e407c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35836/0 (socket says 172.21.15.103:35836) 2024-09-16T01:55:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.955+0000 7fd8e3400640 1 -- 172.21.15.103:0/1670380160 learned_addr learned my addr 172.21.15.103:0/1670380160 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.956+0000 7fd8e3400640 1 -- 172.21.15.103:0/1670380160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e407c930 con 0x7fd8e407de10 2024-09-16T01:55:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.958+0000 7fd8e3400640 1 --2- 172.21.15.103:0/1670380160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e407c270 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009820 tx=0x7fd8d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3585ea36ffc7f73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.958+0000 7fd8e2000640 1 -- 172.21.15.103:0/1670380160 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8d4037070 con 0x7fd8e407de10 2024-09-16T01:55:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.959+0000 7fd8e2000640 1 -- 172.21.15.103:0/1670380160 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd8d4032900 con 0x7fd8e407de10 2024-09-16T01:55:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.959+0000 7fd8e2000640 1 -- 172.21.15.103:0/1670380160 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8d4030850 con 0x7fd8e407de10 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.959+0000 7fd8ea400640 1 -- 172.21.15.103:0/1670380160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 msgr2=0x7fd8e407c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.959+0000 7fd8ea400640 1 --2- 172.21.15.103:0/1670380160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e407c270 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009820 tx=0x7fd8d402a270 comp rx=0 tx=0).stop 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.959+0000 7fd8ea400640 1 -- 172.21.15.103:0/1670380160 shutdown_connections 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.959+0000 7fd8ea400640 1 --2- 172.21.15.103:0/1670380160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e407c270 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.959+0000 7fd8ea400640 1 -- 172.21.15.103:0/1670380160 >> 172.21.15.103:0/1670380160 conn(0x7fd8e4102ff0 msgr2=0x7fd8e4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.959+0000 7fd8ea400640 1 -- 172.21.15.103:0/1670380160 shutdown_connections 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.959+0000 7fd8ea400640 1 -- 172.21.15.103:0/1670380160 wait complete. 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.960+0000 7fd8ea400640 1 Processor -- start 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.961+0000 7fd8ea400640 1 -- start start 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.961+0000 7fd8ea400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.961+0000 7fd8ea400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e410de50 con 0x7fd8e407de10 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.961+0000 7fd8e3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:44.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.961+0000 7fd8e3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e419da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35848/0 (socket says 172.21.15.103:35848) 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.961+0000 7fd8e3400640 1 -- 172.21.15.103:0/872987852 learned_addr learned my addr 172.21.15.103:0/872987852 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.961+0000 7fd8e3400640 1 -- 172.21.15.103:0/872987852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e419dfa0 con 0x7fd8e407de10 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.962+0000 7fd8e3400640 1 --2- 172.21.15.103:0/872987852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e419da60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40097f0 tx=0x7fd8d4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.962+0000 7fd8dbe00640 1 -- 172.21.15.103:0/872987852 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8d4042070 con 0x7fd8e407de10 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.962+0000 7fd8dbe00640 1 -- 172.21.15.103:0/872987852 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd8d4032ca0 con 0x7fd8e407de10 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.962+0000 7fd8ea400640 1 -- 172.21.15.103:0/872987852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e419e230 con 0x7fd8e407de10 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.962+0000 7fd8dbe00640 1 -- 172.21.15.103:0/872987852 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8d4037040 con 0x7fd8e407de10 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.962+0000 7fd8ea400640 1 -- 172.21.15.103:0/872987852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e41a0f20 con 0x7fd8e407de10 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.962+0000 7fd8dbe00640 1 -- 172.21.15.103:0/872987852 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd8d4045070 con 0x7fd8e407de10 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.963+0000 7fd8dbe00640 1 -- 172.21.15.103:0/872987852 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd8d403d430 con 0x7fd8e407de10 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.963+0000 7fd8ea400640 1 -- 172.21.15.103:0/872987852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e407c2f0 con 0x7fd8e407de10 2024-09-16T01:55:44.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:43.966+0000 7fd8dbe00640 1 -- 172.21.15.103:0/872987852 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd8d403db10 con 0x7fd8e407de10 2024-09-16T01:55:44.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:44.029+0000 7fd8ea400640 1 -- 172.21.15.103:0/872987852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fd8e41a1520 con 0x7fd8e407de10 2024-09-16T01:55:44.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:44.031+0000 7fd8dbe00640 1 -- 172.21.15.103:0/872987852 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fd8d403d630 con 0x7fd8e407de10 2024-09-16T01:55:44.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:44.032+0000 7fd8ea400640 1 -- 172.21.15.103:0/872987852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 msgr2=0x7fd8e419da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:44.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:44.032+0000 7fd8ea400640 1 --2- 172.21.15.103:0/872987852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e419da60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40097f0 tx=0x7fd8d4030ec0 comp rx=0 tx=0).stop 2024-09-16T01:55:44.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:44.032+0000 7fd8ea400640 1 -- 172.21.15.103:0/872987852 shutdown_connections 2024-09-16T01:55:44.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:44.032+0000 7fd8ea400640 1 --2- 172.21.15.103:0/872987852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e407de10 0x7fd8e419da60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:44.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:44.032+0000 7fd8ea400640 1 -- 172.21.15.103:0/872987852 >> 172.21.15.103:0/872987852 conn(0x7fd8e4102ff0 msgr2=0x7fd8e4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:44.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:44.033+0000 7fd8ea400640 1 -- 172.21.15.103:0/872987852 shutdown_connections 2024-09-16T01:55:44.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:44.033+0000 7fd8ea400640 1 -- 172.21.15.103:0/872987852 wait complete. 2024-09-16T01:55:44.624 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T01:55:44.641 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T01:55:44.641 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mgr... 2024-09-16T01:55:44.642 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T01:55:44.642 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T01:55:44.985 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.x 2024-09-16T01:55:44.986 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-16T01:55:45.247 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968.target.wants/ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@.service. 2024-09-16T01:55:45.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:45 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/872987852' entity='client.admin' 2024-09-16T01:55:45.572 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:45 smithi103 systemd[1]: Starting Ceph mgr.x for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T01:55:45.828 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:45 smithi103 podman[30619]: 2024-09-16 01:55:45.502580196 +0000 UTC m=+0.027071054 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:55:45.828 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:45 smithi103 podman[30619]: 2024-09-16 01:55:45.617943901 +0000 UTC m=+0.142434738 container create 9e0b235d2a04d65679b8c80e0da3a7f5ccac8daaf062a2a99b95d6912c863652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T01:55:45.828 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:45 smithi103 podman[30619]: 2024-09-16 01:55:45.820526899 +0000 UTC m=+0.345017757 container init 9e0b235d2a04d65679b8c80e0da3a7f5ccac8daaf062a2a99b95d6912c863652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T01:55:45.828 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:45 smithi103 podman[30619]: 2024-09-16 01:55:45.826989211 +0000 UTC m=+0.351480051 container start 9e0b235d2a04d65679b8c80e0da3a7f5ccac8daaf062a2a99b95d6912c863652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:55:45.940 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:55:45.940 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-16T01:55:45.954 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:55:45.954 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-16T01:55:45.954 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-16T01:55:45.955 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T01:55:45.972 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:55:45.972 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-16T01:55:45.987 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:55:45.987 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-16T01:55:45.987 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-16T01:55:45.987 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T01:55:45.987 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr to start... 2024-09-16T01:55:45.987 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr... 2024-09-16T01:55:46.130 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:45 smithi103 bash[30619]: 9e0b235d2a04d65679b8c80e0da3a7f5ccac8daaf062a2a99b95d6912c863652 2024-09-16T01:55:46.130 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:45 smithi103 systemd[1]: Started Ceph mgr.x for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:55:46.130 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:45 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:45.997+0000 7f5ab6bd6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:55:46.130 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:46 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:46.070+0000 7f5ab6bd6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:55:46.383 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:46 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:46.268+0000 7f5ab6bd6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:55:46.383 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:46 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:46.351+0000 7f5ab6bd6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:55:46.717 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:46 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/740507385' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T01:55:46.717 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:46 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:46.535+0000 7f5ab6bd6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:55:46.717 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:46 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:46.716+0000 7f5ab6bd6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:55:47.402 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:47 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:47.121+0000 7f5ab6bd6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:55:47.402 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:47 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:47.193+0000 7f5ab6bd6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:55:47.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:47.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:55:47.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "a20dd7a4-73ce-11ef-bceb-c7b262605968", 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:55:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-16T01:55:47.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T01:55:47.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:47.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T01:55:47.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T01:55:47.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:47.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T01:55:47.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:47.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T01:55:47.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T01:55:47.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T01:55:47.422 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T01:55:47.422 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T01:55:47.422 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T01:55:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T01:55:37:666788+0000", 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T01:55:47.424 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T01:55:37.667595+0000", 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.491+0000 7fe904800640 1 Processor -- start 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.491+0000 7fe904800640 1 -- start start 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.491+0000 7fe904800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc109e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:47.425 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.491+0000 7fe904800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc10a3e0 con 0x7fe8fc109a10 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.492+0000 7fe901e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.492+0000 7fe901e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc109e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35878/0 (socket says 172.21.15.103:35878) 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.492+0000 7fe901e00640 1 -- 172.21.15.103:0/1222204072 learned_addr learned my addr 172.21.15.103:0/1222204072 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.492+0000 7fe901e00640 1 -- 172.21.15.103:0/1222204072 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc10abc0 con 0x7fe8fc109a10 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.493+0000 7fe901e00640 1 --2- 172.21.15.103:0/1222204072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc109e10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009820 tx=0x7fe8f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a4311a249822af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.494+0000 7fe900a00640 1 -- 172.21.15.103:0/1222204072 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f0037070 con 0x7fe8fc109a10 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.494+0000 7fe900a00640 1 -- 172.21.15.103:0/1222204072 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe8f0032900 con 0x7fe8fc109a10 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.494+0000 7fe900a00640 1 -- 172.21.15.103:0/1222204072 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f0030850 con 0x7fe8fc109a10 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.494+0000 7fe904800640 1 -- 172.21.15.103:0/1222204072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 msgr2=0x7fe8fc109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.494+0000 7fe904800640 1 --2- 172.21.15.103:0/1222204072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc109e10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009820 tx=0x7fe8f002a270 comp rx=0 tx=0).stop 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.494+0000 7fe904800640 1 -- 172.21.15.103:0/1222204072 shutdown_connections 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.494+0000 7fe904800640 1 --2- 172.21.15.103:0/1222204072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc109e10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:47.426 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.494+0000 7fe904800640 1 -- 172.21.15.103:0/1222204072 >> 172.21.15.103:0/1222204072 conn(0x7fe8fc07d030 msgr2=0x7fe8fc07d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.495+0000 7fe904800640 1 -- 172.21.15.103:0/1222204072 shutdown_connections 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.495+0000 7fe904800640 1 -- 172.21.15.103:0/1222204072 wait complete. 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.496+0000 7fe904800640 1 Processor -- start 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.496+0000 7fe904800640 1 -- start start 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.496+0000 7fe904800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.496+0000 7fe904800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc10df70 con 0x7fe8fc109a10 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.496+0000 7fe901e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.496+0000 7fe901e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc19da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35888/0 (socket says 172.21.15.103:35888) 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.496+0000 7fe901e00640 1 -- 172.21.15.103:0/740507385 learned_addr learned my addr 172.21.15.103:0/740507385 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.497+0000 7fe901e00640 1 -- 172.21.15.103:0/740507385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc19df90 con 0x7fe8fc109a10 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.497+0000 7fe901e00640 1 --2- 172.21.15.103:0/740507385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc19da50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00097f0 tx=0x7fe8f0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.497+0000 7fe8f6a00640 1 -- 172.21.15.103:0/740507385 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f0042070 con 0x7fe8fc109a10 2024-09-16T01:55:47.427 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.497+0000 7fe8f6a00640 1 -- 172.21.15.103:0/740507385 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe8f0032ca0 con 0x7fe8fc109a10 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.497+0000 7fe8f6a00640 1 -- 172.21.15.103:0/740507385 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f0037040 con 0x7fe8fc109a10 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.497+0000 7fe904800640 1 -- 172.21.15.103:0/740507385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc19e220 con 0x7fe8fc109a10 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.498+0000 7fe904800640 1 -- 172.21.15.103:0/740507385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc1a0f10 con 0x7fe8fc109a10 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.498+0000 7fe8f6a00640 1 -- 172.21.15.103:0/740507385 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8f0045070 con 0x7fe8fc109a10 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.498+0000 7fe8f6a00640 1 -- 172.21.15.103:0/740507385 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe8f0047a30 con 0x7fe8fc109a10 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.499+0000 7fe904800640 1 -- 172.21.15.103:0/740507385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc10e0f0 con 0x7fe8fc109a10 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.501+0000 7fe8f6a00640 1 -- 172.21.15.103:0/740507385 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe8f003da60 con 0x7fe8fc109a10 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.565+0000 7fe904800640 1 -- 172.21.15.103:0/740507385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe8fc1a13f0 con 0x7fe8fc109a10 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.565+0000 7fe8f6a00640 1 -- 172.21.15.103:0/740507385 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe8f003d430 con 0x7fe8fc109a10 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.566+0000 7fe904800640 1 -- 172.21.15.103:0/740507385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 msgr2=0x7fe8fc19da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.566+0000 7fe904800640 1 --2- 172.21.15.103:0/740507385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc19da50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00097f0 tx=0x7fe8f0030ec0 comp rx=0 tx=0).stop 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.567+0000 7fe904800640 1 -- 172.21.15.103:0/740507385 shutdown_connections 2024-09-16T01:55:47.428 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.567+0000 7fe904800640 1 --2- 172.21.15.103:0/740507385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8fc109a10 0x7fe8fc19da50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:47.429 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.567+0000 7fe904800640 1 -- 172.21.15.103:0/740507385 >> 172.21.15.103:0/740507385 conn(0x7fe8fc07d030 msgr2=0x7fe8fc107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:47.429 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.567+0000 7fe904800640 1 -- 172.21.15.103:0/740507385 shutdown_connections 2024-09-16T01:55:47.429 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:46.567+0000 7fe904800640 1 -- 172.21.15.103:0/740507385 wait complete. 2024-09-16T01:55:47.429 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (1/15)... 2024-09-16T01:55:47.754 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:47 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:47.407+0000 7f5ab6bd6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:55:49.004 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:48 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:48.550+0000 7f5ab6bd6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:55:49.004 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:48 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:48.629+0000 7f5ab6bd6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:55:49.849 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:49 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:49.570+0000 7f5ab6bd6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:55:49.849 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:49 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:49.717+0000 7f5ab6bd6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:55:50.141 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:49 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2304438418' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T01:55:50.141 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:49 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:49.889+0000 7f5ab6bd6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:55:50.141 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:49 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:49.965+0000 7f5ab6bd6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:55:50.141 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:50.140+0000 7f5ab6bd6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:55:50.485 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:50.226+0000 7f5ab6bd6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:55:50.486 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:50.299+0000 7f5ab6bd6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:55:50.486 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:50.484+0000 7f5ab6bd6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:55:50.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:50.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:55:50.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "a20dd7a4-73ce-11ef-bceb-c7b262605968", 2024-09-16T01:55:50.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T01:55:50.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T01:55:50.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T01:55:50.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T01:55:50.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T01:55:50.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:50.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T01:55:50.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:50.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T01:55:50.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T01:55:50.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T01:55:50.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T01:55:50.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T01:55:50.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T01:55:50.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:50.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T01:55:37:666788+0000", 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:50.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T01:55:37.667595+0000", 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.871+0000 7f8dafe00640 1 Processor -- start 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.871+0000 7f8dafe00640 1 -- start start 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.871+0000 7f8dafe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da8107c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.871+0000 7f8dafe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81081d0 con 0x7f8da8107800 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.872+0000 7f8dad400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da8107c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:50.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.872+0000 7f8dad400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da8107c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35944/0 (socket says 172.21.15.103:35944) 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.872+0000 7f8dad400640 1 -- 172.21.15.103:0/901995724 learned_addr learned my addr 172.21.15.103:0/901995724 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.872+0000 7f8dad400640 1 -- 172.21.15.103:0/901995724 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da8108a00 con 0x7f8da8107800 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.873+0000 7f8dad400640 1 --2- 172.21.15.103:0/901995724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da8107c00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009820 tx=0x7f8d9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8506758ec243b92 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.873+0000 7f8da7e00640 1 -- 172.21.15.103:0/901995724 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d9c037070 con 0x7f8da8107800 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.873+0000 7f8da7e00640 1 -- 172.21.15.103:0/901995724 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d9c032900 con 0x7f8da8107800 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.873+0000 7f8da7e00640 1 -- 172.21.15.103:0/901995724 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d9c030850 con 0x7f8da8107800 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.874+0000 7f8dafe00640 1 -- 172.21.15.103:0/901995724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 msgr2=0x7f8da8107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.874+0000 7f8dafe00640 1 --2- 172.21.15.103:0/901995724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da8107c00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009820 tx=0x7f8d9c02a270 comp rx=0 tx=0).stop 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.874+0000 7f8dafe00640 1 -- 172.21.15.103:0/901995724 shutdown_connections 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.874+0000 7f8dafe00640 1 --2- 172.21.15.103:0/901995724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da8107c00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.874+0000 7f8dafe00640 1 -- 172.21.15.103:0/901995724 >> 172.21.15.103:0/901995724 conn(0x7f8da8102ff0 msgr2=0x7f8da8105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.874+0000 7f8dafe00640 1 -- 172.21.15.103:0/901995724 shutdown_connections 2024-09-16T01:55:50.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.874+0000 7f8dafe00640 1 -- 172.21.15.103:0/901995724 wait complete. 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.875+0000 7f8dafe00640 1 Processor -- start 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.876+0000 7f8dafe00640 1 -- start start 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.876+0000 7f8dafe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da807db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.876+0000 7f8dafe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8da810bdb0 con 0x7f8da8107800 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.876+0000 7f8dad400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da807db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.876+0000 7f8dad400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da807db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35946/0 (socket says 172.21.15.103:35946) 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.876+0000 7f8dad400640 1 -- 172.21.15.103:0/2304438418 learned_addr learned my addr 172.21.15.103:0/2304438418 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.877+0000 7f8dad400640 1 -- 172.21.15.103:0/2304438418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da807e060 con 0x7f8da8107800 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.877+0000 7f8dad400640 1 --2- 172.21.15.103:0/2304438418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da807db20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0097f0 tx=0x7f8d9c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.877+0000 7f8da6000640 1 -- 172.21.15.103:0/2304438418 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d9c042070 con 0x7f8da8107800 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.877+0000 7f8da6000640 1 -- 172.21.15.103:0/2304438418 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d9c032ca0 con 0x7f8da8107800 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.877+0000 7f8dafe00640 1 -- 172.21.15.103:0/2304438418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da807c250 con 0x7f8da8107800 2024-09-16T01:55:50.558 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.877+0000 7f8da6000640 1 -- 172.21.15.103:0/2304438418 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d9c037040 con 0x7f8da8107800 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.877+0000 7f8dafe00640 1 -- 172.21.15.103:0/2304438418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da807c790 con 0x7f8da8107800 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.878+0000 7f8da6000640 1 -- 172.21.15.103:0/2304438418 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d9c045070 con 0x7f8da8107800 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.878+0000 7f8da6000640 1 -- 172.21.15.103:0/2304438418 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8d9c047ae0 con 0x7f8da8107800 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.878+0000 7f8dafe00640 1 -- 172.21.15.103:0/2304438418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da8107c80 con 0x7f8da8107800 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.881+0000 7f8da6000640 1 -- 172.21.15.103:0/2304438418 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8d9c03da60 con 0x7f8da8107800 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.945+0000 7f8dafe00640 1 -- 172.21.15.103:0/2304438418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8da807ccc0 con 0x7f8da8107800 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.945+0000 7f8da6000640 1 -- 172.21.15.103:0/2304438418 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f8d9c03d430 con 0x7f8da8107800 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.946+0000 7f8dafe00640 1 -- 172.21.15.103:0/2304438418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 msgr2=0x7f8da807db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.946+0000 7f8dafe00640 1 --2- 172.21.15.103:0/2304438418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da807db20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0097f0 tx=0x7f8d9c030ec0 comp rx=0 tx=0).stop 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.946+0000 7f8dafe00640 1 -- 172.21.15.103:0/2304438418 shutdown_connections 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.946+0000 7f8dafe00640 1 --2- 172.21.15.103:0/2304438418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8107800 0x7f8da807db20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.946+0000 7f8dafe00640 1 -- 172.21.15.103:0/2304438418 >> 172.21.15.103:0/2304438418 conn(0x7f8da8102ff0 msgr2=0x7f8da8103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:50.559 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.946+0000 7f8dafe00640 1 -- 172.21.15.103:0/2304438418 shutdown_connections 2024-09-16T01:55:50.560 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:49.946+0000 7f8dafe00640 1 -- 172.21.15.103:0/2304438418 wait complete. 2024-09-16T01:55:50.560 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (2/15)... 2024-09-16T01:55:50.745 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:50.566+0000 7f5ab6bd6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:55:50.745 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:50.647+0000 7f5ab6bd6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:55:50.745 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:50.744+0000 7f5ab6bd6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:55:51.004 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:50.815+0000 7f5ab6bd6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:55:51.004 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:50.886+0000 7f5ab6bd6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:55:52.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:51.789+0000 7f5ab6bd6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:55:52.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: Activating manager daemon x 2024-09-16T01:55:52.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: mgrmap e2: x(active, starting, since 0.00393749s) 2024-09-16T01:55:52.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: from='mgr.14100 172.21.15.103:0/1883340624' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:55:52.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: from='mgr.14100 172.21.15.103:0/1883340624' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:55:52.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: from='mgr.14100 172.21.15.103:0/1883340624' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:55:52.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: from='mgr.14100 172.21.15.103:0/1883340624' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:55:52.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: from='mgr.14100 172.21.15.103:0/1883340624' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:55:52.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: Manager daemon x is now available 2024-09-16T01:55:52.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: from='mgr.14100 172.21.15.103:0/1883340624' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:55:52.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: from='mgr.14100 172.21.15.103:0/1883340624' entity='mgr.x' 2024-09-16T01:55:52.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: from='mgr.14100 172.21.15.103:0/1883340624' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:55:52.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: from='mgr.14100 172.21.15.103:0/1883340624' entity='mgr.x' 2024-09-16T01:55:52.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:51 smithi103 ceph-mon[30398]: from='mgr.14100 172.21.15.103:0/1883340624' entity='mgr.x' 2024-09-16T01:55:54.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:53 smithi103 ceph-mon[30398]: mgrmap e3: x(active, since 1.00783s) 2024-09-16T01:55:54.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:53 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/376001082' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T01:55:54.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:54.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:55:54.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "a20dd7a4-73ce-11ef-bceb-c7b262605968", 2024-09-16T01:55:54.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T01:55:54.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T01:55:54.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T01:55:54.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T01:55:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T01:55:54.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:54.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T01:55:54.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:54.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T01:55:54.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T01:55:54.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T01:55:54.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T01:55:54.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T01:55:54.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T01:55:54.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:54.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T01:55:54.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T01:55:37:666788+0000", 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T01:55:54.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:55:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T01:55:37.667595+0000", 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.025+0000 7fe7f1a00640 1 Processor -- start 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.025+0000 7fe7f1a00640 1 -- start start 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.025+0000 7fe7f1a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.025+0000 7fe7f1a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec108000 con 0x7fe7ec107630 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.025+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.026+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36006/0 (socket says 172.21.15.103:36006) 2024-09-16T01:55:54.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.026+0000 7fe7ebe00640 1 -- 172.21.15.103:0/3849822770 learned_addr learned my addr 172.21.15.103:0/3849822770 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:54.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.026+0000 7fe7ebe00640 1 -- 172.21.15.103:0/3849822770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec108830 con 0x7fe7ec107630 2024-09-16T01:55:54.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.027+0000 7fe7ebe00640 1 --2- 172.21.15.103:0/3849822770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec107a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009820 tx=0x7fe7dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5fc58b61c63c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:54.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.027+0000 7fe7eb400640 1 -- 172.21.15.103:0/3849822770 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc037070 con 0x7fe7ec107630 2024-09-16T01:55:54.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.027+0000 7fe7eb400640 1 -- 172.21.15.103:0/3849822770 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7dc032900 con 0x7fe7ec107630 2024-09-16T01:55:54.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.027+0000 7fe7eb400640 1 -- 172.21.15.103:0/3849822770 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc030850 con 0x7fe7ec107630 2024-09-16T01:55:54.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.028+0000 7fe7f1a00640 1 -- 172.21.15.103:0/3849822770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 msgr2=0x7fe7ec107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:54.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.028+0000 7fe7f1a00640 1 --2- 172.21.15.103:0/3849822770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec107a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009820 tx=0x7fe7dc02a270 comp rx=0 tx=0).stop 2024-09-16T01:55:54.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.028+0000 7fe7f1a00640 1 -- 172.21.15.103:0/3849822770 shutdown_connections 2024-09-16T01:55:54.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.028+0000 7fe7f1a00640 1 --2- 172.21.15.103:0/3849822770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec107a30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:54.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.028+0000 7fe7f1a00640 1 -- 172.21.15.103:0/3849822770 >> 172.21.15.103:0/3849822770 conn(0x7fe7ec102dc0 msgr2=0x7fe7ec105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.028+0000 7fe7f1a00640 1 -- 172.21.15.103:0/3849822770 shutdown_connections 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.028+0000 7fe7f1a00640 1 -- 172.21.15.103:0/3849822770 wait complete. 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.029+0000 7fe7f1a00640 1 Processor -- start 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.029+0000 7fe7f1a00640 1 -- start start 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.029+0000 7fe7f1a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec07db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.029+0000 7fe7f1a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec10bbe0 con 0x7fe7ec107630 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.030+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec07db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.030+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec07db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36022/0 (socket says 172.21.15.103:36022) 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.030+0000 7fe7ebe00640 1 -- 172.21.15.103:0/376001082 learned_addr learned my addr 172.21.15.103:0/376001082 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.030+0000 7fe7ebe00640 1 -- 172.21.15.103:0/376001082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec07e060 con 0x7fe7ec107630 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.030+0000 7fe7ebe00640 1 --2- 172.21.15.103:0/376001082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec07db20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0097f0 tx=0x7fe7dc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.031+0000 7fe7e9600640 1 -- 172.21.15.103:0/376001082 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc037040 con 0x7fe7ec107630 2024-09-16T01:55:54.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.031+0000 7fe7e9600640 1 -- 172.21.15.103:0/376001082 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7dc040070 con 0x7fe7ec107630 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.031+0000 7fe7f1a00640 1 -- 172.21.15.103:0/376001082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec07c250 con 0x7fe7ec107630 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.031+0000 7fe7e9600640 1 -- 172.21.15.103:0/376001082 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc03a390 con 0x7fe7ec107630 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.031+0000 7fe7f1a00640 1 -- 172.21.15.103:0/376001082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec07c730 con 0x7fe7ec107630 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.032+0000 7fe7e9600640 1 -- 172.21.15.103:0/376001082 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7fe7dc03b430 con 0x7fe7ec107630 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.032+0000 7fe7f1a00640 1 -- 172.21.15.103:0/376001082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ec107ab0 con 0x7fe7ec107630 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.032+0000 7fe7e9600640 1 --2- 172.21.15.103:0/376001082 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7fe7d00390a0 0x7fe7d003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.032+0000 7fe7e9600640 1 -- 172.21.15.103:0/376001082 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe7dc06c170 con 0x7fe7ec107630 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.033+0000 7fe7e3e00640 1 --2- 172.21.15.103:0/376001082 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7fe7d00390a0 0x7fe7d003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.033+0000 7fe7e3e00640 1 --2- 172.21.15.103:0/376001082 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7fe7d00390a0 0x7fe7d003b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40098c0 tx=0x7fe7d4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.038+0000 7fe7e9600640 1 -- 172.21.15.103:0/376001082 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7dc032900 con 0x7fe7ec107630 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.275+0000 7fe7f1a00640 1 -- 172.21.15.103:0/376001082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe7ec07d010 con 0x7fe7ec107630 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.276+0000 7fe7e9600640 1 -- 172.21.15.103:0/376001082 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fe7dc043c30 con 0x7fe7ec107630 2024-09-16T01:55:54.074 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.279+0000 7fe7f1a00640 1 -- 172.21.15.103:0/376001082 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7fe7d00390a0 msgr2=0x7fe7d003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.279+0000 7fe7f1a00640 1 --2- 172.21.15.103:0/376001082 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7fe7d00390a0 0x7fe7d003b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40098c0 tx=0x7fe7d4006db0 comp rx=0 tx=0).stop 2024-09-16T01:55:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.279+0000 7fe7f1a00640 1 -- 172.21.15.103:0/376001082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 msgr2=0x7fe7ec07db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.279+0000 7fe7f1a00640 1 --2- 172.21.15.103:0/376001082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec07db20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0097f0 tx=0x7fe7dc030d80 comp rx=0 tx=0).stop 2024-09-16T01:55:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.279+0000 7fe7f1a00640 1 -- 172.21.15.103:0/376001082 shutdown_connections 2024-09-16T01:55:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.279+0000 7fe7f1a00640 1 --2- 172.21.15.103:0/376001082 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7fe7d00390a0 0x7fe7d003b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.279+0000 7fe7f1a00640 1 --2- 172.21.15.103:0/376001082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec107630 0x7fe7ec07db20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.279+0000 7fe7f1a00640 1 -- 172.21.15.103:0/376001082 >> 172.21.15.103:0/376001082 conn(0x7fe7ec102dc0 msgr2=0x7fe7ec103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.279+0000 7fe7f1a00640 1 -- 172.21.15.103:0/376001082 shutdown_connections 2024-09-16T01:55:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:53.279+0000 7fe7f1a00640 1 -- 172.21.15.103:0/376001082 wait complete. 2024-09-16T01:55:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:mgr is available 2024-09-16T01:55:55.091 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:54 smithi103 ceph-mon[30398]: mgrmap e4: x(active, since 2s) 2024-09-16T01:55:55.091 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:54 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/44200470' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T01:55:55.327 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.533+0000 7f334d200640 1 Processor -- start 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.533+0000 7f334d200640 1 -- start start 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.533+0000 7f334d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f3348107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.533+0000 7f334d200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3348108000 con 0x7f3348107630 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.533+0000 7f3347400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f3348107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:55.328 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.534+0000 7f3347400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f3348107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36026/0 (socket says 172.21.15.103:36026) 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.534+0000 7f3347400640 1 -- 172.21.15.103:0/236219790 learned_addr learned my addr 172.21.15.103:0/236219790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.534+0000 7f3347400640 1 -- 172.21.15.103:0/236219790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3348108830 con 0x7f3348107630 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.535+0000 7f3347400640 1 --2- 172.21.15.103:0/236219790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f3348107a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3338009820 tx=0x7f333802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352321dcd67c9ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.535+0000 7f3346a00640 1 -- 172.21.15.103:0/236219790 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3338037070 con 0x7f3348107630 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.535+0000 7f3346a00640 1 -- 172.21.15.103:0/236219790 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3338032900 con 0x7f3348107630 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.535+0000 7f3346a00640 1 -- 172.21.15.103:0/236219790 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3338030850 con 0x7f3348107630 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.536+0000 7f334d200640 1 -- 172.21.15.103:0/236219790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 msgr2=0x7f3348107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.536+0000 7f334d200640 1 --2- 172.21.15.103:0/236219790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f3348107a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3338009820 tx=0x7f333802a270 comp rx=0 tx=0).stop 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.536+0000 7f334d200640 1 -- 172.21.15.103:0/236219790 shutdown_connections 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.536+0000 7f334d200640 1 --2- 172.21.15.103:0/236219790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f3348107a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.536+0000 7f334d200640 1 -- 172.21.15.103:0/236219790 >> 172.21.15.103:0/236219790 conn(0x7f3348102dc0 msgr2=0x7f3348105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:55.329 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.536+0000 7f334d200640 1 -- 172.21.15.103:0/236219790 shutdown_connections 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.536+0000 7f334d200640 1 -- 172.21.15.103:0/236219790 wait complete. 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.537+0000 7f334d200640 1 Processor -- start 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.537+0000 7f334d200640 1 -- start start 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.537+0000 7f334d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f334819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.537+0000 7f334d200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f334810bbe0 con 0x7f3348107630 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.538+0000 7f3347400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f334819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.538+0000 7f3347400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f334819d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36038/0 (socket says 172.21.15.103:36038) 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.538+0000 7f3347400640 1 -- 172.21.15.103:0/44200470 learned_addr learned my addr 172.21.15.103:0/44200470 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.538+0000 7f3347400640 1 -- 172.21.15.103:0/44200470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334819dd50 con 0x7f3348107630 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.538+0000 7f3347400640 1 --2- 172.21.15.103:0/44200470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f334819d810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f33380097f0 tx=0x7f3338030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.538+0000 7f3344c00640 1 -- 172.21.15.103:0/44200470 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3338037040 con 0x7f3348107630 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.538+0000 7f3344c00640 1 -- 172.21.15.103:0/44200470 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3338040070 con 0x7f3348107630 2024-09-16T01:55:55.330 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.538+0000 7f3344c00640 1 -- 172.21.15.103:0/44200470 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f333803a390 con 0x7f3348107630 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.538+0000 7f334d200640 1 -- 172.21.15.103:0/44200470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334819dfe0 con 0x7f3348107630 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.539+0000 7f334d200640 1 -- 172.21.15.103:0/44200470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33481a0cd0 con 0x7f3348107630 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.540+0000 7f3344c00640 1 -- 172.21.15.103:0/44200470 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f333803b430 con 0x7f3348107630 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.540+0000 7f334d200640 1 -- 172.21.15.103:0/44200470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3348107ab0 con 0x7f3348107630 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.540+0000 7f3344c00640 1 --2- 172.21.15.103:0/44200470 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f332c039160 0x7f332c03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.540+0000 7f3344c00640 1 -- 172.21.15.103:0/44200470 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f333806c3c0 con 0x7f3348107630 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.540+0000 7f333fe00640 1 --2- 172.21.15.103:0/44200470 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f332c039160 0x7f332c03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.541+0000 7f333fe00640 1 --2- 172.21.15.103:0/44200470 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f332c039160 0x7f332c03b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f33300098c0 tx=0x7f3330006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.545+0000 7f3344c00640 1 -- 172.21.15.103:0/44200470 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f333806c120 con 0x7f3348107630 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.721+0000 7f334d200640 1 -- 172.21.15.103:0/44200470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f334819e460 con 0x7f3348107630 2024-09-16T01:55:55.331 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.723+0000 7f3344c00640 1 -- 172.21.15.103:0/44200470 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f3338040ce0 con 0x7f3348107630 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.725+0000 7f334d200640 1 -- 172.21.15.103:0/44200470 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f332c039160 msgr2=0x7f332c03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.725+0000 7f334d200640 1 --2- 172.21.15.103:0/44200470 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f332c039160 0x7f332c03b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f33300098c0 tx=0x7f3330006db0 comp rx=0 tx=0).stop 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.725+0000 7f334d200640 1 -- 172.21.15.103:0/44200470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 msgr2=0x7f334819d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.725+0000 7f334d200640 1 --2- 172.21.15.103:0/44200470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f334819d810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f33380097f0 tx=0x7f3338030d80 comp rx=0 tx=0).stop 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.725+0000 7f334d200640 1 -- 172.21.15.103:0/44200470 shutdown_connections 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.725+0000 7f334d200640 1 --2- 172.21.15.103:0/44200470 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f332c039160 0x7f332c03b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.725+0000 7f334d200640 1 --2- 172.21.15.103:0/44200470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3348107630 0x7f334819d810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.725+0000 7f334d200640 1 -- 172.21.15.103:0/44200470 >> 172.21.15.103:0/44200470 conn(0x7f3348102dc0 msgr2=0x7f3348103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.725+0000 7f334d200640 1 -- 172.21.15.103:0/44200470 shutdown_connections 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:54.725+0000 7f334d200640 1 -- 172.21.15.103:0/44200470 wait complete. 2024-09-16T01:55:55.332 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling cephadm module... 2024-09-16T01:55:56.312 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:56 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2131226888' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T01:55:56.313 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: ignoring --setuser ceph since I am not root 2024-09-16T01:55:56.313 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: ignoring --setgroup ceph since I am not root 2024-09-16T01:55:56.313 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:56.311+0000 7f82087d4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:55:56.568 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:56.382+0000 7f82087d4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:55:56.568 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:56.567+0000 7f82087d4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:55:56.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.801+0000 7f5150c00640 1 Processor -- start 2024-09-16T01:55:56.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.801+0000 7f5150c00640 1 -- start start 2024-09-16T01:55:56.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.801+0000 7f5150c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c109e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:56.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.801+0000 7f5150c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f514c10a3e0 con 0x7f514c109a10 2024-09-16T01:55:56.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.802+0000 7f5149e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:56.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.802+0000 7f5149e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c109e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36040/0 (socket says 172.21.15.103:36040) 2024-09-16T01:55:56.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.802+0000 7f5149e00640 1 -- 172.21.15.103:0/1756600510 learned_addr learned my addr 172.21.15.103:0/1756600510 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.802+0000 7f5149e00640 1 -- 172.21.15.103:0/1756600510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c10abc0 con 0x7f514c109a10 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.803+0000 7f5149e00640 1 --2- 172.21.15.103:0/1756600510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c109e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f513c009820 tx=0x7f513c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2c407f3dd08fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.803+0000 7f5148a00640 1 -- 172.21.15.103:0/1756600510 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f513c037070 con 0x7f514c109a10 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.803+0000 7f5148a00640 1 -- 172.21.15.103:0/1756600510 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f513c032900 con 0x7f514c109a10 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.803+0000 7f5148a00640 1 -- 172.21.15.103:0/1756600510 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f513c030850 con 0x7f514c109a10 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.804+0000 7f5150c00640 1 -- 172.21.15.103:0/1756600510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 msgr2=0x7f514c109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.804+0000 7f5150c00640 1 --2- 172.21.15.103:0/1756600510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c109e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f513c009820 tx=0x7f513c02a270 comp rx=0 tx=0).stop 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.804+0000 7f5150c00640 1 -- 172.21.15.103:0/1756600510 shutdown_connections 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.804+0000 7f5150c00640 1 --2- 172.21.15.103:0/1756600510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c109e10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.804+0000 7f5150c00640 1 -- 172.21.15.103:0/1756600510 >> 172.21.15.103:0/1756600510 conn(0x7f514c07d030 msgr2=0x7f514c07d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.804+0000 7f5150c00640 1 -- 172.21.15.103:0/1756600510 shutdown_connections 2024-09-16T01:55:56.779 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.804+0000 7f5150c00640 1 -- 172.21.15.103:0/1756600510 wait complete. 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.805+0000 7f5150c00640 1 Processor -- start 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.805+0000 7f5150c00640 1 -- start start 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.805+0000 7f5150c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.805+0000 7f5150c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f514c10df70 con 0x7f514c109a10 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.806+0000 7f5149e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.806+0000 7f5149e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c19da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36042/0 (socket says 172.21.15.103:36042) 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.806+0000 7f5149e00640 1 -- 172.21.15.103:0/2131226888 learned_addr learned my addr 172.21.15.103:0/2131226888 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.806+0000 7f5149e00640 1 -- 172.21.15.103:0/2131226888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c19df90 con 0x7f514c109a10 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.806+0000 7f5149e00640 1 --2- 172.21.15.103:0/2131226888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c19da50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f513c009940 tx=0x7f513c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.806+0000 7f5142a00640 1 -- 172.21.15.103:0/2131226888 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f513c042070 con 0x7f514c109a10 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.806+0000 7f5142a00640 1 -- 172.21.15.103:0/2131226888 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f513c032be0 con 0x7f514c109a10 2024-09-16T01:55:56.780 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.807+0000 7f5150c00640 1 -- 172.21.15.103:0/2131226888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514c19e220 con 0x7f514c109a10 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.807+0000 7f5142a00640 1 -- 172.21.15.103:0/2131226888 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f513c037040 con 0x7f514c109a10 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.807+0000 7f5150c00640 1 -- 172.21.15.103:0/2131226888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514c1a0f10 con 0x7f514c109a10 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.808+0000 7f5142a00640 1 -- 172.21.15.103:0/2131226888 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f513c045070 con 0x7f514c109a10 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.808+0000 7f5150c00640 1 -- 172.21.15.103:0/2131226888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514c10e0f0 con 0x7f514c109a10 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.808+0000 7f5142a00640 1 --2- 172.21.15.103:0/2131226888 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f51300391b0 0x7f513003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.808+0000 7f5142a00640 1 -- 172.21.15.103:0/2131226888 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f513c06c260 con 0x7f514c109a10 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.809+0000 7f5149400640 1 --2- 172.21.15.103:0/2131226888 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f51300391b0 0x7f513003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.809+0000 7f5149400640 1 --2- 172.21.15.103:0/2131226888 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f51300391b0 0x7f513003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f51380098c0 tx=0x7f5138006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:55.813+0000 7f5142a00640 1 -- 172.21.15.103:0/2131226888 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f513c045360 con 0x7f514c109a10 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.037+0000 7f5150c00640 1 -- 172.21.15.103:0/2131226888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f514c1a1580 con 0x7f514c109a10 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.089+0000 7f5142a00640 1 -- 172.21.15.103:0/2131226888 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f513c06bc80 con 0x7f514c109a10 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.091+0000 7f5142a00640 1 -- 172.21.15.103:0/2131226888 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f513c03cb00 con 0x7f514c109a10 2024-09-16T01:55:56.781 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.092+0000 7f5150c00640 1 -- 172.21.15.103:0/2131226888 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f51300391b0 msgr2=0x7f513003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:56.782 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.092+0000 7f5150c00640 1 --2- 172.21.15.103:0/2131226888 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f51300391b0 0x7f513003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f51380098c0 tx=0x7f5138006db0 comp rx=0 tx=0).stop 2024-09-16T01:55:56.782 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.092+0000 7f5150c00640 1 -- 172.21.15.103:0/2131226888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 msgr2=0x7f514c19da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:56.782 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.092+0000 7f5150c00640 1 --2- 172.21.15.103:0/2131226888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c19da50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f513c009940 tx=0x7f513c0326e0 comp rx=0 tx=0).stop 2024-09-16T01:55:56.782 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.092+0000 7f5150c00640 1 -- 172.21.15.103:0/2131226888 shutdown_connections 2024-09-16T01:55:56.782 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.092+0000 7f5150c00640 1 --2- 172.21.15.103:0/2131226888 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f51300391b0 0x7f513003b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:56.782 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.092+0000 7f5150c00640 1 --2- 172.21.15.103:0/2131226888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f514c109a10 0x7f514c19da50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:56.782 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.092+0000 7f5150c00640 1 -- 172.21.15.103:0/2131226888 >> 172.21.15.103:0/2131226888 conn(0x7f514c07d030 msgr2=0x7f514c107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:56.782 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.092+0000 7f5150c00640 1 -- 172.21.15.103:0/2131226888 shutdown_connections 2024-09-16T01:55:56.782 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:56.092+0000 7f5150c00640 1 -- 172.21.15.103:0/2131226888 wait complete. 2024-09-16T01:55:56.827 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:56.648+0000 7f82087d4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:55:57.095 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:56.825+0000 7f82087d4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:55:57.095 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:56.993+0000 7f82087d4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:55:57.379 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:57 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2131226888' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T01:55:57.379 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:57 smithi103 ceph-mon[30398]: mgrmap e5: x(active, since 4s) 2024-09-16T01:55:57.672 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:57.378+0000 7f82087d4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:55:57.672 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:57.451+0000 7f82087d4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:55:57.672 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:57.671+0000 7f82087d4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:55:58.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:55:58.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T01:55:58.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T01:55:58.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T01:55:58.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T01:55:58.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:55:58.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.314+0000 7f4db8000640 1 Processor -- start 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.314+0000 7f4db8000640 1 -- start start 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.315+0000 7f4db8000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.315+0000 7f4db8000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4db007c8c0 con 0x7f4db007de40 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.315+0000 7f4db5600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.315+0000 7f4db5600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36078/0 (socket says 172.21.15.103:36078) 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.315+0000 7f4db5600640 1 -- 172.21.15.103:0/2006327419 learned_addr learned my addr 172.21.15.103:0/2006327419 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.315+0000 7f4db5600640 1 -- 172.21.15.103:0/2006327419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db007ca40 con 0x7f4db007de40 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.316+0000 7f4db5600640 1 --2- 172.21.15.103:0/2006327419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db007c2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009820 tx=0x7f4da402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218b43bef5ecb645 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.317+0000 7f4dafe00640 1 -- 172.21.15.103:0/2006327419 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da4037070 con 0x7f4db007de40 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.317+0000 7f4dafe00640 1 -- 172.21.15.103:0/2006327419 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4da4032900 con 0x7f4db007de40 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.317+0000 7f4dafe00640 1 -- 172.21.15.103:0/2006327419 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da4030850 con 0x7f4db007de40 2024-09-16T01:55:58.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.318+0000 7f4db8000640 1 -- 172.21.15.103:0/2006327419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 msgr2=0x7f4db007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.318+0000 7f4db8000640 1 --2- 172.21.15.103:0/2006327419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db007c2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009820 tx=0x7f4da402a270 comp rx=0 tx=0).stop 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.318+0000 7f4db8000640 1 -- 172.21.15.103:0/2006327419 shutdown_connections 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.318+0000 7f4db8000640 1 --2- 172.21.15.103:0/2006327419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db007c2f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.318+0000 7f4db8000640 1 -- 172.21.15.103:0/2006327419 >> 172.21.15.103:0/2006327419 conn(0x7f4db0103000 msgr2=0x7f4db0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.318+0000 7f4db8000640 1 -- 172.21.15.103:0/2006327419 shutdown_connections 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.318+0000 7f4db8000640 1 -- 172.21.15.103:0/2006327419 wait complete. 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.319+0000 7f4db8000640 1 Processor -- start 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.319+0000 7f4db8000640 1 -- start start 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.319+0000 7f4db8000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db01a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.319+0000 7f4db8000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01099f0 con 0x7f4db007de40 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.320+0000 7f4db5600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db01a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.320+0000 7f4db5600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db01a1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36080/0 (socket says 172.21.15.103:36080) 2024-09-16T01:55:58.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.320+0000 7f4db5600640 1 -- 172.21.15.103:0/3512310554 learned_addr learned my addr 172.21.15.103:0/3512310554 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.320+0000 7f4db5600640 1 -- 172.21.15.103:0/3512310554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db01a2310 con 0x7f4db007de40 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.320+0000 7f4db5600640 1 --2- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db01a1dd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4da40097f0 tx=0x7f4da4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.320+0000 7f4dae000640 1 -- 172.21.15.103:0/3512310554 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da4037040 con 0x7f4db007de40 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.321+0000 7f4dae000640 1 -- 172.21.15.103:0/3512310554 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4da4040070 con 0x7f4db007de40 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.321+0000 7f4dae000640 1 -- 172.21.15.103:0/3512310554 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da403a390 con 0x7f4db007de40 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.321+0000 7f4db8000640 1 -- 172.21.15.103:0/3512310554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db01a25a0 con 0x7f4db007de40 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.321+0000 7f4db8000640 1 -- 172.21.15.103:0/3512310554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db01a5290 con 0x7f4db007de40 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.322+0000 7f4db8000640 1 -- 172.21.15.103:0/3512310554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db007c370 con 0x7f4db007de40 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.322+0000 7f4dae000640 1 -- 172.21.15.103:0/3512310554 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f4da403b430 con 0x7f4db007de40 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.323+0000 7f4dae000640 1 --2- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f4d98039200 0x7f4d9803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.323+0000 7f4dae000640 1 -- 172.21.15.103:0/3512310554 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4da406c430 con 0x7f4db007de40 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.323+0000 7f4db4c00640 1 -- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f4d98039200 msgr2=0x7f4d9803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1247798496 2024-09-16T01:55:58.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.323+0000 7f4db4c00640 1 --2- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f4d98039200 0x7f4d9803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.332+0000 7f4dae000640 1 -- 172.21.15.103:0/3512310554 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4da406c190 con 0x7f4db007de40 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.524+0000 7f4db4c00640 1 -- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f4d98039200 msgr2=0x7f4d9803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1247798496 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.524+0000 7f4db4c00640 1 --2- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f4d98039200 0x7f4d9803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.559+0000 7f4db8000640 1 -- 172.21.15.103:0/3512310554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4db019b240 con 0x7f4db007de40 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.559+0000 7f4dae000640 1 -- 172.21.15.103:0/3512310554 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f4da4032900 con 0x7f4db007de40 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.562+0000 7f4db8000640 1 -- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f4d98039200 msgr2=0x7f4d9803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.562+0000 7f4db8000640 1 --2- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f4d98039200 0x7f4d9803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.562+0000 7f4db8000640 1 -- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 msgr2=0x7f4db01a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.562+0000 7f4db8000640 1 --2- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db01a1dd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4da40097f0 tx=0x7f4da4030d80 comp rx=0 tx=0).stop 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.562+0000 7f4db8000640 1 -- 172.21.15.103:0/3512310554 shutdown_connections 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.562+0000 7f4db8000640 1 --2- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f4d98039200 0x7f4d9803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:58.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.562+0000 7f4db8000640 1 --2- 172.21.15.103:0/3512310554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db007de40 0x7f4db01a1dd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:58.263 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.562+0000 7f4db8000640 1 -- 172.21.15.103:0/3512310554 >> 172.21.15.103:0/3512310554 conn(0x7f4db0103000 msgr2=0x7f4db01053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:58.263 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.562+0000 7f4db8000640 1 -- 172.21.15.103:0/3512310554 shutdown_connections 2024-09-16T01:55:58.263 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:57.562+0000 7f4db8000640 1 -- 172.21.15.103:0/3512310554 wait complete. 2024-09-16T01:55:58.263 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-09-16T01:55:58.263 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 5... 2024-09-16T01:55:58.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:55:58 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3512310554' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T01:55:59.004 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:58 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:58.798+0000 7f82087d4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:55:59.004 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:58 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:58.877+0000 7f82087d4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:56:00.079 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:59 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:59.783+0000 7f82087d4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:56:00.079 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:55:59 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:55:59.917+0000 7f82087d4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:56:00.079 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:00 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:00.078+0000 7f82087d4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:56:00.400 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:00 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:00.149+0000 7f82087d4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:56:00.400 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:00 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:00.318+0000 7f82087d4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:56:00.401 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:00 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:00.399+0000 7f82087d4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:56:00.725 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:00 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:00.470+0000 7f82087d4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:56:00.726 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:00 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:00.648+0000 7f82087d4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:56:00.726 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:00 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:00.724+0000 7f82087d4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:56:01.004 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:00 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:00.801+0000 7f82087d4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:56:01.004 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:00 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:00.897+0000 7f82087d4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:56:01.005 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:00 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:00.967+0000 7f82087d4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:56:01.504 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:01.036+0000 7f82087d4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:56:02.004 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:01.934+0000 7f82087d4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:56:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: Active manager daemon x restarted 2024-09-16T01:56:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: Activating manager daemon x 2024-09-16T01:56:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T01:56:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: mgrmap e6: x(active, starting, since 0.00393532s) 2024-09-16T01:56:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:56:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:56:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:56:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:56:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:56:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: Manager daemon x is now available 2024-09-16T01:56:02.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:02.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:02.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:02.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:01 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:03.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:02 smithi103 ceph-mon[30398]: Found migration_current of "None". Setting to last migration. 2024-09-16T01:56:03.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:02 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:56:03.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:02 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:56:03.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:02 smithi103 ceph-mon[30398]: mgrmap e7: x(active, since 1.00682s) 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.733+0000 7f578f800640 1 Processor -- start 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.733+0000 7f578f800640 1 -- start start 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.734+0000 7f578f800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f5788109e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.734+0000 7f578f800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f578810a3d0 con 0x7f5788109a00 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.734+0000 7f578ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f5788109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.734+0000 7f578ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f5788109e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53314/0 (socket says 172.21.15.103:53314) 2024-09-16T01:56:03.552 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.734+0000 7f578ce00640 1 -- 172.21.15.103:0/2331593127 learned_addr learned my addr 172.21.15.103:0/2331593127 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.734+0000 7f578ce00640 1 -- 172.21.15.103:0/2331593127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578810ac00 con 0x7f5788109a00 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.735+0000 7f578ce00640 1 --2- 172.21.15.103:0/2331593127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f5788109e00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f577c009820 tx=0x7f577c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25889fdfc34e3ee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.735+0000 7f5787400640 1 -- 172.21.15.103:0/2331593127 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f577c037070 con 0x7f5788109a00 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.736+0000 7f5787400640 1 -- 172.21.15.103:0/2331593127 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f577c032900 con 0x7f5788109a00 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.736+0000 7f5787400640 1 -- 172.21.15.103:0/2331593127 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f577c030850 con 0x7f5788109a00 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.736+0000 7f578f800640 1 -- 172.21.15.103:0/2331593127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 msgr2=0x7f5788109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.736+0000 7f578f800640 1 --2- 172.21.15.103:0/2331593127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f5788109e00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f577c009820 tx=0x7f577c02a270 comp rx=0 tx=0).stop 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.736+0000 7f578f800640 1 -- 172.21.15.103:0/2331593127 shutdown_connections 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.736+0000 7f578f800640 1 --2- 172.21.15.103:0/2331593127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f5788109e00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.736+0000 7f578f800640 1 -- 172.21.15.103:0/2331593127 >> 172.21.15.103:0/2331593127 conn(0x7f578807cfe0 msgr2=0x7f578807d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.736+0000 7f578f800640 1 -- 172.21.15.103:0/2331593127 shutdown_connections 2024-09-16T01:56:03.553 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.737+0000 7f578f800640 1 -- 172.21.15.103:0/2331593127 wait complete. 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.737+0000 7f578f800640 1 Processor -- start 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.738+0000 7f578f800640 1 -- start start 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.738+0000 7f578f800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f578819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.738+0000 7f578f800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f578810dfb0 con 0x7f5788109a00 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.738+0000 7f578ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f578819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.738+0000 7f578ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f578819da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53330/0 (socket says 172.21.15.103:53330) 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.738+0000 7f578ce00640 1 -- 172.21.15.103:0/1819036505 learned_addr learned my addr 172.21.15.103:0/1819036505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.739+0000 7f578ce00640 1 -- 172.21.15.103:0/1819036505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578819df40 con 0x7f5788109a00 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.739+0000 7f578ce00640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f578819da00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f577c009940 tx=0x7f577c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.739+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f577c042070 con 0x7f5788109a00 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.739+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f577c0308c0 con 0x7f5788109a00 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.739+0000 7f578f800640 1 -- 172.21.15.103:0/1819036505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578819e1d0 con 0x7f5788109a00 2024-09-16T01:56:03.554 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.739+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f577c037040 con 0x7f5788109a00 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.739+0000 7f578f800640 1 -- 172.21.15.103:0/1819036505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57881a0ec0 con 0x7f5788109a00 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.740+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f577c045070 con 0x7f5788109a00 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.741+0000 7f5785600640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 0x7f577003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.741+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 --> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f577003bd80 con 0x7f57700391b0 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.741+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f577c06c350 con 0x7f5788109a00 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.741+0000 7f5787e00640 1 -- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 msgr2=0x7f577003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1247798496 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.741+0000 7f5787e00640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 0x7f577003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.942+0000 7f5787e00640 1 -- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 msgr2=0x7f577003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1247798496 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:58.942+0000 7f5787e00640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 0x7f577003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:59.343+0000 7f5787e00640 1 -- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 msgr2=0x7f577003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1247798496 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:55:59.343+0000 7f5787e00640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 0x7f577003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:00.144+0000 7f5787e00640 1 -- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 msgr2=0x7f577003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1247798496 2024-09-16T01:56:03.555 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:00.144+0000 7f5787e00640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 0x7f577003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:01.745+0000 7f5787e00640 1 -- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 msgr2=0x7f577003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1247798496 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:01.745+0000 7f5787e00640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 0x7f577003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:01.939+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f577c03c870 con 0x7f5788109a00 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:01.939+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 msgr2=0x7f577003b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:01.939+0000 7f5785600640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 0x7f577003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.940+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f577c045c40 con 0x7f5788109a00 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.940+0000 7f5785600640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f577003cae0 0x7f577003eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.940+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 --> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f577c02bba0 con 0x7f577003cae0 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.941+0000 7f5787e00640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f577003cae0 0x7f577003eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.942+0000 7f5787e00640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f577003cae0 0x7f577003eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5774003980 tx=0x7f5774009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.943+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 <== mgr.14118 v2:172.21.15.103:6800/623307529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f577c02bba0 con 0x7f577003cae0 2024-09-16T01:56:03.556 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.950+0000 7f578f800640 1 -- 172.21.15.103:0/1819036505 --> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f578810e130 con 0x7f577003cae0 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.951+0000 7f5785600640 1 -- 172.21.15.103:0/1819036505 <== mgr.14118 v2:172.21.15.103:6800/623307529 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f578810e130 con 0x7f577003cae0 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.951+0000 7f578f800640 1 -- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f577003cae0 msgr2=0x7f577003eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.951+0000 7f578f800640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f577003cae0 0x7f577003eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5774003980 tx=0x7f5774009290 comp rx=0 tx=0).stop 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.951+0000 7f578f800640 1 -- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 msgr2=0x7f578819da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.951+0000 7f578f800640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f578819da00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f577c009940 tx=0x7f577c032ce0 comp rx=0 tx=0).stop 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.951+0000 7f578f800640 1 -- 172.21.15.103:0/1819036505 shutdown_connections 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.951+0000 7f578f800640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f577003cae0 0x7f577003eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.951+0000 7f578f800640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:6800/1247798496,v1:172.21.15.103:6801/1247798496] conn(0x7f57700391b0 0x7f577003b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.951+0000 7f578f800640 1 --2- 172.21.15.103:0/1819036505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788109a00 0x7f578819da00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.951+0000 7f578f800640 1 -- 172.21.15.103:0/1819036505 >> 172.21.15.103:0/1819036505 conn(0x7f578807cfe0 msgr2=0x7f5788107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.952+0000 7f578f800640 1 -- 172.21.15.103:0/1819036505 shutdown_connections 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:02.952+0000 7f578f800640 1 -- 172.21.15.103:0/1819036505 wait complete. 2024-09-16T01:56:03.557 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 5 is available 2024-09-16T01:56:03.558 INFO:teuthology.orchestra.run.smithi103.stdout:Setting orchestrator backend to cephadm... 2024-09-16T01:56:04.385 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-mon[30398]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T01:56:04.385 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-mon[30398]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T01:56:04.385 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:04.385 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:04.755 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:04.383+0000 7f8160000640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:01:56:04] ENGINE Error in HTTPServer.serve 2024-09-16T01:56:04.755 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: Traceback (most recent call last): 2024-09-16T01:56:04.755 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:56:04.755 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: self._connections.run(self.expiration_interval) 2024-09-16T01:56:04.755 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: self._run(expiration_interval) 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: s = self.context.wrap_socket( 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: return self.sslsocket_class._create( 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: self.do_handshake() 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:56:04.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: self._sslobj.do_handshake() 2024-09-16T01:56:04.757 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:56:04.757 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:04 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:04.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.037+0000 7f8067e00640 1 Processor -- start 2024-09-16T01:56:04.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.038+0000 7f8067e00640 1 -- start start 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.038+0000 7f8067e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f8068107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.038+0000 7f8067e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8068108000 con 0x7f8068107630 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.038+0000 7f8066a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f8068107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.038+0000 7f8066a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f8068107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53402/0 (socket says 172.21.15.103:53402) 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.038+0000 7f8066a00640 1 -- 172.21.15.103:0/1596875110 learned_addr learned my addr 172.21.15.103:0/1596875110 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.039+0000 7f8066a00640 1 -- 172.21.15.103:0/1596875110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8068108830 con 0x7f8068107630 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.040+0000 7f8066a00640 1 --2- 172.21.15.103:0/1596875110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f8068107a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8058009820 tx=0x7f805802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706cb732a12d87a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.040+0000 7f8065600640 1 -- 172.21.15.103:0/1596875110 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8058037070 con 0x7f8068107630 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.040+0000 7f8065600640 1 -- 172.21.15.103:0/1596875110 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8058032900 con 0x7f8068107630 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.040+0000 7f8065600640 1 -- 172.21.15.103:0/1596875110 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8058030850 con 0x7f8068107630 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.041+0000 7f8067e00640 1 -- 172.21.15.103:0/1596875110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 msgr2=0x7f8068107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.041+0000 7f8067e00640 1 --2- 172.21.15.103:0/1596875110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f8068107a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8058009820 tx=0x7f805802a270 comp rx=0 tx=0).stop 2024-09-16T01:56:04.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.041+0000 7f8067e00640 1 -- 172.21.15.103:0/1596875110 shutdown_connections 2024-09-16T01:56:04.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.041+0000 7f8067e00640 1 --2- 172.21.15.103:0/1596875110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f8068107a30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:04.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.041+0000 7f8067e00640 1 -- 172.21.15.103:0/1596875110 >> 172.21.15.103:0/1596875110 conn(0x7f8068102dc0 msgr2=0x7f8068105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:04.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.041+0000 7f8067e00640 1 -- 172.21.15.103:0/1596875110 shutdown_connections 2024-09-16T01:56:04.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.041+0000 7f8067e00640 1 -- 172.21.15.103:0/1596875110 wait complete. 2024-09-16T01:56:04.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.042+0000 7f8067e00640 1 Processor -- start 2024-09-16T01:56:04.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.042+0000 7f8067e00640 1 -- start start 2024-09-16T01:56:04.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.042+0000 7f8067e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f806819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:04.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.042+0000 7f8067e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f806810bbe0 con 0x7f8068107630 2024-09-16T01:56:04.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.043+0000 7f8066a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f806819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.043+0000 7f8066a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f806819d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53418/0 (socket says 172.21.15.103:53418) 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.043+0000 7f8066a00640 1 -- 172.21.15.103:0/1678085321 learned_addr learned my addr 172.21.15.103:0/1678085321 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.043+0000 7f8066a00640 1 -- 172.21.15.103:0/1678085321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806819dd60 con 0x7f8068107630 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.043+0000 7f8066a00640 1 --2- 172.21.15.103:0/1678085321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f806819d820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f80580097f0 tx=0x7f8058030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.043+0000 7f805f400640 1 -- 172.21.15.103:0/1678085321 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8058037040 con 0x7f8068107630 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.044+0000 7f805f400640 1 -- 172.21.15.103:0/1678085321 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8058040070 con 0x7f8068107630 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.044+0000 7f805f400640 1 -- 172.21.15.103:0/1678085321 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f805803a390 con 0x7f8068107630 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.044+0000 7f8067e00640 1 -- 172.21.15.103:0/1678085321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806819dff0 con 0x7f8068107630 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.044+0000 7f8067e00640 1 -- 172.21.15.103:0/1678085321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80681a0ce0 con 0x7f8068107630 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.045+0000 7f805f400640 1 -- 172.21.15.103:0/1678085321 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f8058032900 con 0x7f8068107630 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.045+0000 7f8067e00640 1 -- 172.21.15.103:0/1678085321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8068107ab0 con 0x7f8068107630 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.045+0000 7f805f400640 1 --2- 172.21.15.103:0/1678085321 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f804c0390f0 0x7f804c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:04.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.045+0000 7f805f400640 1 -- 172.21.15.103:0/1678085321 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f805806c5f0 con 0x7f8068107630 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.045+0000 7f8066000640 1 --2- 172.21.15.103:0/1678085321 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f804c0390f0 0x7f804c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.046+0000 7f8066000640 1 --2- 172.21.15.103:0/1678085321 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f804c0390f0 0x7f804c03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80540098c0 tx=0x7f8054006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.051+0000 7f805f400640 1 -- 172.21.15.103:0/1678085321 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8058040ce0 con 0x7f8068107630 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.132+0000 7f805f400640 1 -- 172.21.15.103:0/1678085321 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8058044ca0 con 0x7f8068107630 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.248+0000 7f8067e00640 1 -- 172.21.15.103:0/1678085321 --> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8068104000 con 0x7f804c0390f0 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.254+0000 7f805f400640 1 -- 172.21.15.103:0/1678085321 <== mgr.14118 v2:172.21.15.103:6800/623307529 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8068104000 con 0x7f804c0390f0 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.256+0000 7f8067e00640 1 -- 172.21.15.103:0/1678085321 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f804c0390f0 msgr2=0x7f804c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.256+0000 7f8067e00640 1 --2- 172.21.15.103:0/1678085321 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f804c0390f0 0x7f804c03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80540098c0 tx=0x7f8054006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.256+0000 7f8067e00640 1 -- 172.21.15.103:0/1678085321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 msgr2=0x7f806819d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.256+0000 7f8067e00640 1 --2- 172.21.15.103:0/1678085321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f806819d820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f80580097f0 tx=0x7f8058030d80 comp rx=0 tx=0).stop 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.256+0000 7f8067e00640 1 -- 172.21.15.103:0/1678085321 shutdown_connections 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.256+0000 7f8067e00640 1 --2- 172.21.15.103:0/1678085321 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f804c0390f0 0x7f804c03b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:04.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.256+0000 7f8067e00640 1 --2- 172.21.15.103:0/1678085321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8068107630 0x7f806819d820 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:04.916 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.256+0000 7f8067e00640 1 -- 172.21.15.103:0/1678085321 >> 172.21.15.103:0/1678085321 conn(0x7f8068102dc0 msgr2=0x7f80681038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:04.916 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.256+0000 7f8067e00640 1 -- 172.21.15.103:0/1678085321 shutdown_connections 2024-09-16T01:56:04.916 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:04.256+0000 7f8067e00640 1 -- 172.21.15.103:0/1678085321 wait complete. 2024-09-16T01:56:05.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:05 smithi103 ceph-mon[30398]: mgrmap e8: x(active, since 2s) 2024-09-16T01:56:05.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:05 smithi103 ceph-mon[30398]: [16/Sep/2024:01:56:04] ENGINE Bus STARTING 2024-09-16T01:56:05.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:05 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:05.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:05 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:05.155 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:05 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:06.188 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:06 smithi103 ceph-mon[30398]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:06.188 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:06 smithi103 ceph-mon[30398]: [16/Sep/2024:01:56:04] ENGINE Serving on http://172.21.15.103:8765 2024-09-16T01:56:06.188 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:06 smithi103 ceph-mon[30398]: [16/Sep/2024:01:56:04] ENGINE Serving on https://172.21.15.103:7150 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:06 smithi103 ceph-mon[30398]: [16/Sep/2024:01:56:04] ENGINE Bus STARTED 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:06 smithi103 ceph-mon[30398]: [16/Sep/2024:01:56:04] ENGINE Error in HTTPServer.serve 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:56:06.189 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-16T01:56:06.190 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:56:06.190 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-16T01:56:06.190 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:56:06.190 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-16T01:56:06.190 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:56:06.190 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-16T01:56:06.190 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:56:06.252 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T01:56:06.252 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.398+0000 7fb513c00640 1 Processor -- start 2024-09-16T01:56:06.252 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.398+0000 7fb513c00640 1 -- start start 2024-09-16T01:56:06.252 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.398+0000 7fb513c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c107be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.252 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.399+0000 7fb513c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1081b0 con 0x7fb50c1077e0 2024-09-16T01:56:06.252 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.399+0000 7fb511200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c107be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.252 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.399+0000 7fb511200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c107be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53428/0 (socket says 172.21.15.103:53428) 2024-09-16T01:56:06.253 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.399+0000 7fb511200640 1 -- 172.21.15.103:0/3319277999 learned_addr learned my addr 172.21.15.103:0/3319277999 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:06.253 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.399+0000 7fb511200640 1 -- 172.21.15.103:0/3319277999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c1089e0 con 0x7fb50c1077e0 2024-09-16T01:56:06.253 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.400+0000 7fb511200640 1 --2- 172.21.15.103:0/3319277999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c107be0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009820 tx=0x7fb4fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf65baf9023073e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:06.253 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.400+0000 7fb50b400640 1 -- 172.21.15.103:0/3319277999 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4fc037070 con 0x7fb50c1077e0 2024-09-16T01:56:06.253 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.400+0000 7fb50b400640 1 -- 172.21.15.103:0/3319277999 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4fc032900 con 0x7fb50c1077e0 2024-09-16T01:56:06.253 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.400+0000 7fb50b400640 1 -- 172.21.15.103:0/3319277999 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4fc030850 con 0x7fb50c1077e0 2024-09-16T01:56:06.253 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.401+0000 7fb513c00640 1 -- 172.21.15.103:0/3319277999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 msgr2=0x7fb50c107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:06.253 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.401+0000 7fb513c00640 1 --2- 172.21.15.103:0/3319277999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c107be0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009820 tx=0x7fb4fc02a270 comp rx=0 tx=0).stop 2024-09-16T01:56:06.253 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.401+0000 7fb513c00640 1 -- 172.21.15.103:0/3319277999 shutdown_connections 2024-09-16T01:56:06.253 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.401+0000 7fb513c00640 1 --2- 172.21.15.103:0/3319277999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c107be0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.401+0000 7fb513c00640 1 -- 172.21.15.103:0/3319277999 >> 172.21.15.103:0/3319277999 conn(0x7fb50c102ff0 msgr2=0x7fb50c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.401+0000 7fb513c00640 1 -- 172.21.15.103:0/3319277999 shutdown_connections 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.401+0000 7fb513c00640 1 -- 172.21.15.103:0/3319277999 wait complete. 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.402+0000 7fb513c00640 1 Processor -- start 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.402+0000 7fb513c00640 1 -- start start 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.403+0000 7fb513c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c07db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.403+0000 7fb513c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c10bd90 con 0x7fb50c1077e0 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.403+0000 7fb511200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c07db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.403+0000 7fb511200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c07db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53440/0 (socket says 172.21.15.103:53440) 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.403+0000 7fb511200640 1 -- 172.21.15.103:0/723736651 learned_addr learned my addr 172.21.15.103:0/723736651 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.403+0000 7fb511200640 1 -- 172.21.15.103:0/723736651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c07e060 con 0x7fb50c1077e0 2024-09-16T01:56:06.254 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.404+0000 7fb511200640 1 --2- 172.21.15.103:0/723736651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c07db20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0097f0 tx=0x7fb4fc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.404+0000 7fb509600640 1 -- 172.21.15.103:0/723736651 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4fc037040 con 0x7fb50c1077e0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.404+0000 7fb509600640 1 -- 172.21.15.103:0/723736651 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4fc040070 con 0x7fb50c1077e0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.404+0000 7fb513c00640 1 -- 172.21.15.103:0/723736651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50c07c250 con 0x7fb50c1077e0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.404+0000 7fb509600640 1 -- 172.21.15.103:0/723736651 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4fc03a390 con 0x7fb50c1077e0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.404+0000 7fb513c00640 1 -- 172.21.15.103:0/723736651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50c07c730 con 0x7fb50c1077e0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.405+0000 7fb513c00640 1 -- 172.21.15.103:0/723736651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50c107c60 con 0x7fb50c1077e0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.405+0000 7fb509600640 1 -- 172.21.15.103:0/723736651 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb4fc03b430 con 0x7fb50c1077e0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.406+0000 7fb509600640 1 --2- 172.21.15.103:0/723736651 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb4f0039200 0x7fb4f003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.406+0000 7fb509600640 1 -- 172.21.15.103:0/723736651 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb4fc06c7d0 con 0x7fb50c1077e0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.406+0000 7fb50be00640 1 --2- 172.21.15.103:0/723736651 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb4f0039200 0x7fb4f003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.407+0000 7fb50be00640 1 --2- 172.21.15.103:0/723736651 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb4f0039200 0x7fb4f003b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80098c0 tx=0x7fb4f8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.411+0000 7fb509600640 1 -- 172.21.15.103:0/723736651 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4fc06c120 con 0x7fb50c1077e0 2024-09-16T01:56:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.585+0000 7fb513c00640 1 -- 172.21.15.103:0/723736651 --> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb50c104280 con 0x7fb4f0039200 2024-09-16T01:56:06.256 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.586+0000 7fb509600640 1 -- 172.21.15.103:0/723736651 <== mgr.14118 v2:172.21.15.103:6800/623307529 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb50c104280 con 0x7fb4f0039200 2024-09-16T01:56:06.256 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.588+0000 7fb513c00640 1 -- 172.21.15.103:0/723736651 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb4f0039200 msgr2=0x7fb4f003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:06.256 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.588+0000 7fb513c00640 1 --2- 172.21.15.103:0/723736651 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb4f0039200 0x7fb4f003b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80098c0 tx=0x7fb4f8006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.256 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.588+0000 7fb513c00640 1 -- 172.21.15.103:0/723736651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 msgr2=0x7fb50c07db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:06.256 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.588+0000 7fb513c00640 1 --2- 172.21.15.103:0/723736651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c07db20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0097f0 tx=0x7fb4fc030d80 comp rx=0 tx=0).stop 2024-09-16T01:56:06.256 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.588+0000 7fb513c00640 1 -- 172.21.15.103:0/723736651 shutdown_connections 2024-09-16T01:56:06.256 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.588+0000 7fb513c00640 1 --2- 172.21.15.103:0/723736651 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb4f0039200 0x7fb4f003b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.256 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.588+0000 7fb513c00640 1 --2- 172.21.15.103:0/723736651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c1077e0 0x7fb50c07db20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.256 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.588+0000 7fb513c00640 1 -- 172.21.15.103:0/723736651 >> 172.21.15.103:0/723736651 conn(0x7fb50c102ff0 msgr2=0x7fb50c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:06.256 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.588+0000 7fb513c00640 1 -- 172.21.15.103:0/723736651 shutdown_connections 2024-09-16T01:56:06.257 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:05.588+0000 7fb513c00640 1 -- 172.21.15.103:0/723736651 wait complete. 2024-09-16T01:56:06.257 INFO:teuthology.orchestra.run.smithi103.stdout:Generating ssh key... 2024-09-16T01:56:07.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-mon[30398]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:07.681 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: Generating public/private rsa key pair. 2024-09-16T01:56:07.681 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: Your identification has been saved in /tmp/tmpzqop0tsb/key 2024-09-16T01:56:07.681 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: Your public key has been saved in /tmp/tmpzqop0tsb/key.pub 2024-09-16T01:56:07.681 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: The key fingerprint is: 2024-09-16T01:56:07.681 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: SHA256:eA2W6fwu96QezEmAh9FreLk+NItuFnJuuUXrwt77GtM ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:56:07.681 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: The key's randomart image is: 2024-09-16T01:56:07.681 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: +---[RSA 3072]----+ 2024-09-16T01:56:07.681 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: | .. | 2024-09-16T01:56:07.681 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: | +.o | 2024-09-16T01:56:07.681 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: | o.Bo | 2024-09-16T01:56:07.682 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: | .*=+ | 2024-09-16T01:56:07.684 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: | .oS.o | 2024-09-16T01:56:07.684 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: | . oo+B . | 2024-09-16T01:56:07.684 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: | = *=oE . | 2024-09-16T01:56:07.684 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: | X+=+.+ | 2024-09-16T01:56:07.685 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: | *+o=O=.. | 2024-09-16T01:56:07.685 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: +----[SHA256]-----+ 2024-09-16T01:56:07.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.703+0000 7f8e35000640 1 Processor -- start 2024-09-16T01:56:07.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.703+0000 7f8e35000640 1 -- start start 2024-09-16T01:56:07.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.704+0000 7f8e35000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e30107c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.704+0000 7f8e35000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301081d0 con 0x7f8e30107800 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.704+0000 7f8e2de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e30107c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.704+0000 7f8e2de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e30107c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53444/0 (socket says 172.21.15.103:53444) 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.704+0000 7f8e2de00640 1 -- 172.21.15.103:0/3647505882 learned_addr learned my addr 172.21.15.103:0/3647505882 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.704+0000 7f8e2de00640 1 -- 172.21.15.103:0/3647505882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e30108a00 con 0x7f8e30107800 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.705+0000 7f8e2de00640 1 --2- 172.21.15.103:0/3647505882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e30107c00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009820 tx=0x7f8e2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3626f58796e80a83 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.705+0000 7f8e2d400640 1 -- 172.21.15.103:0/3647505882 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e20037070 con 0x7f8e30107800 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.706+0000 7f8e2d400640 1 -- 172.21.15.103:0/3647505882 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e20032900 con 0x7f8e30107800 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.706+0000 7f8e2d400640 1 -- 172.21.15.103:0/3647505882 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e20030850 con 0x7f8e30107800 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.706+0000 7f8e35000640 1 -- 172.21.15.103:0/3647505882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 msgr2=0x7f8e30107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:07.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.706+0000 7f8e35000640 1 --2- 172.21.15.103:0/3647505882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e30107c00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009820 tx=0x7f8e2002a270 comp rx=0 tx=0).stop 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.706+0000 7f8e35000640 1 -- 172.21.15.103:0/3647505882 shutdown_connections 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.706+0000 7f8e35000640 1 --2- 172.21.15.103:0/3647505882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e30107c00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.706+0000 7f8e35000640 1 -- 172.21.15.103:0/3647505882 >> 172.21.15.103:0/3647505882 conn(0x7f8e30102ff0 msgr2=0x7f8e30105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.706+0000 7f8e35000640 1 -- 172.21.15.103:0/3647505882 shutdown_connections 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.706+0000 7f8e35000640 1 -- 172.21.15.103:0/3647505882 wait complete. 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.707+0000 7f8e35000640 1 Processor -- start 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.707+0000 7f8e35000640 1 -- start start 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.708+0000 7f8e35000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e3007db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.708+0000 7f8e35000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3010bdb0 con 0x7f8e30107800 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.708+0000 7f8e2de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e3007db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.708+0000 7f8e2de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e3007db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53460/0 (socket says 172.21.15.103:53460) 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.708+0000 7f8e2de00640 1 -- 172.21.15.103:0/4101850283 learned_addr learned my addr 172.21.15.103:0/4101850283 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.708+0000 7f8e2de00640 1 -- 172.21.15.103:0/4101850283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3007e060 con 0x7f8e30107800 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.708+0000 7f8e2de00640 1 --2- 172.21.15.103:0/4101850283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e3007db20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8e200097f0 tx=0x7f8e20030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.709+0000 7f8e27400640 1 -- 172.21.15.103:0/4101850283 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e20042070 con 0x7f8e30107800 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.709+0000 7f8e27400640 1 -- 172.21.15.103:0/4101850283 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e20032ca0 con 0x7f8e30107800 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.709+0000 7f8e35000640 1 -- 172.21.15.103:0/4101850283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3007c250 con 0x7f8e30107800 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.709+0000 7f8e27400640 1 -- 172.21.15.103:0/4101850283 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e20037040 con 0x7f8e30107800 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.709+0000 7f8e35000640 1 -- 172.21.15.103:0/4101850283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3007c790 con 0x7f8e30107800 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.710+0000 7f8e27400640 1 -- 172.21.15.103:0/4101850283 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8e20045070 con 0x7f8e30107800 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.710+0000 7f8e35000640 1 -- 172.21.15.103:0/4101850283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e30107c80 con 0x7f8e30107800 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.710+0000 7f8e27400640 1 --2- 172.21.15.103:0/4101850283 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f8e140391b0 0x7f8e1403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.711+0000 7f8e27400640 1 -- 172.21.15.103:0/4101850283 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8e2006c1a0 con 0x7f8e30107800 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.711+0000 7f8e25400640 1 --2- 172.21.15.103:0/4101850283 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f8e140391b0 0x7f8e1403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.711+0000 7f8e25400640 1 --2- 172.21.15.103:0/4101850283 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f8e140391b0 0x7f8e1403b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e180098c0 tx=0x7f8e18006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.715+0000 7f8e27400640 1 -- 172.21.15.103:0/4101850283 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e2004b170 con 0x7f8e30107800 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:06.889+0000 7f8e35000640 1 -- 172.21.15.103:0/4101850283 --> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8e30104380 con 0x7f8e140391b0 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.354+0000 7f8e27400640 1 -- 172.21.15.103:0/4101850283 <== mgr.14118 v2:172.21.15.103:6800/623307529 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8e30104380 con 0x7f8e140391b0 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.356+0000 7f8e35000640 1 -- 172.21.15.103:0/4101850283 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f8e140391b0 msgr2=0x7f8e1403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.356+0000 7f8e35000640 1 --2- 172.21.15.103:0/4101850283 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f8e140391b0 0x7f8e1403b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e180098c0 tx=0x7f8e18006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.356+0000 7f8e35000640 1 -- 172.21.15.103:0/4101850283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 msgr2=0x7f8e3007db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.356+0000 7f8e35000640 1 --2- 172.21.15.103:0/4101850283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e3007db20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8e200097f0 tx=0x7f8e20030ec0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.356+0000 7f8e35000640 1 -- 172.21.15.103:0/4101850283 shutdown_connections 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.356+0000 7f8e35000640 1 --2- 172.21.15.103:0/4101850283 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f8e140391b0 0x7f8e1403b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.357+0000 7f8e35000640 1 --2- 172.21.15.103:0/4101850283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e30107800 0x7f8e3007db20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.357+0000 7f8e35000640 1 -- 172.21.15.103:0/4101850283 >> 172.21.15.103:0/4101850283 conn(0x7f8e30102ff0 msgr2=0x7f8e30103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.357+0000 7f8e35000640 1 -- 172.21.15.103:0/4101850283 shutdown_connections 2024-09-16T01:56:07.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:07.357+0000 7f8e35000640 1 -- 172.21.15.103:0/4101850283 wait complete. 2024-09-16T01:56:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:08 smithi103 ceph-mon[30398]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:08 smithi103 ceph-mon[30398]: Generating ssh key... 2024-09-16T01:56:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:08 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:08 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:09.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4qx/0LzvxSJ/kCPWrmyYgdq7IMssgpFGEAIacf9wKdAnlqkTR42Jw8uOPo9FVjlErBu5Ju2Gwz5EKlzPIRgpxMurgnV9HFxPp71Lj+p3KAkJr+SArtUIzPRbcnysSdNLuyoTFpzSKTzYTq7W8FoOvE+onZiGBev2RytTasvMEddQOerNjA7BzUs86amGC1Rc/KDq1s/VxFAQf4h9YPhPxhm5yTHDsGFeyofu+D+CmnQJtBK1aCfQujHgwz7IZaK08H5Ys+NLKlFr1vLkISPcz/Htw+XWbrEkOTzbmci2oFiiV1gWzad9xnG67TPRr+qK1D59KQtnEgnwRT74u0Su6kwa+JGYCdeHyeb8EcP5RaZKJtM3tFyiFMS+1p6roCMYqohN+hhiTYui3Nq6CrsiDlo8OlZTR28YP2gUk1qsnh0PjATInXVDFx7Ob95W96+9Af0TyjJUumJPA81BZq5gIp/lljAgBviTIpf0scqgubUJhkILsy1nYPsvHw7UXtp0= ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:56:09.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.428+0000 7f03df200640 1 Processor -- start 2024-09-16T01:56:09.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.428+0000 7f03df200640 1 -- start start 2024-09-16T01:56:09.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.428+0000 7f03df200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e0107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:09.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.428+0000 7f03df200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0108000 con 0x7f03e0107630 2024-09-16T01:56:09.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.429+0000 7f03dde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e0107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:09.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.429+0000 7f03dde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e0107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41102/0 (socket says 172.21.15.103:41102) 2024-09-16T01:56:09.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.429+0000 7f03dde00640 1 -- 172.21.15.103:0/3298791457 learned_addr learned my addr 172.21.15.103:0/3298791457 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:09.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.429+0000 7f03dde00640 1 -- 172.21.15.103:0/3298791457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e0108830 con 0x7f03e0107630 2024-09-16T01:56:09.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.430+0000 7f03dde00640 1 --2- 172.21.15.103:0/3298791457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e0107a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009820 tx=0x7f03d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7be1c89659df6b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:09.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.430+0000 7f03dca00640 1 -- 172.21.15.103:0/3298791457 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03d0037070 con 0x7f03e0107630 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.430+0000 7f03dca00640 1 -- 172.21.15.103:0/3298791457 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03d0032900 con 0x7f03e0107630 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.430+0000 7f03dca00640 1 -- 172.21.15.103:0/3298791457 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03d0030850 con 0x7f03e0107630 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.431+0000 7f03df200640 1 -- 172.21.15.103:0/3298791457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 msgr2=0x7f03e0107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.431+0000 7f03df200640 1 --2- 172.21.15.103:0/3298791457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e0107a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009820 tx=0x7f03d002a270 comp rx=0 tx=0).stop 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.431+0000 7f03df200640 1 -- 172.21.15.103:0/3298791457 shutdown_connections 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.431+0000 7f03df200640 1 --2- 172.21.15.103:0/3298791457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e0107a30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.431+0000 7f03df200640 1 -- 172.21.15.103:0/3298791457 >> 172.21.15.103:0/3298791457 conn(0x7f03e0102dc0 msgr2=0x7f03e0105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.431+0000 7f03df200640 1 -- 172.21.15.103:0/3298791457 shutdown_connections 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.431+0000 7f03df200640 1 -- 172.21.15.103:0/3298791457 wait complete. 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.432+0000 7f03df200640 1 Processor -- start 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.432+0000 7f03df200640 1 -- start start 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.433+0000 7f03df200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:09.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.433+0000 7f03df200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03e010bbe0 con 0x7f03e0107630 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.433+0000 7f03dde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.433+0000 7f03dde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e019d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41110/0 (socket says 172.21.15.103:41110) 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.433+0000 7f03dde00640 1 -- 172.21.15.103:0/3122209592 learned_addr learned my addr 172.21.15.103:0/3122209592 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.433+0000 7f03dde00640 1 -- 172.21.15.103:0/3122209592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e019dd50 con 0x7f03e0107630 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.433+0000 7f03dde00640 1 --2- 172.21.15.103:0/3122209592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e019d810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009940 tx=0x7f03d0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.434+0000 7f03d6a00640 1 -- 172.21.15.103:0/3122209592 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03d0042070 con 0x7f03e0107630 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.434+0000 7f03d6a00640 1 -- 172.21.15.103:0/3122209592 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03d00308c0 con 0x7f03e0107630 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.434+0000 7f03d6a00640 1 -- 172.21.15.103:0/3122209592 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03d0037040 con 0x7f03e0107630 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.434+0000 7f03df200640 1 -- 172.21.15.103:0/3122209592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e019dfe0 con 0x7f03e0107630 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.434+0000 7f03df200640 1 -- 172.21.15.103:0/3122209592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e01a0cd0 con 0x7f03e0107630 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.435+0000 7f03d6a00640 1 -- 172.21.15.103:0/3122209592 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f03d0045070 con 0x7f03e0107630 2024-09-16T01:56:09.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.435+0000 7f03df200640 1 -- 172.21.15.103:0/3122209592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e0107ab0 con 0x7f03e0107630 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.435+0000 7f03d6a00640 1 --2- 172.21.15.103:0/3122209592 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f03c4039200 0x7f03c403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.435+0000 7f03d6a00640 1 -- 172.21.15.103:0/3122209592 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f03d006c3e0 con 0x7f03e0107630 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.436+0000 7f03dd400640 1 --2- 172.21.15.103:0/3122209592 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f03c4039200 0x7f03c403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.436+0000 7f03dd400640 1 --2- 172.21.15.103:0/3122209592 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f03c4039200 0x7f03c403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0098c0 tx=0x7f03cc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.440+0000 7f03d6a00640 1 -- 172.21.15.103:0/3122209592 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03d004b170 con 0x7f03e0107630 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.617+0000 7f03df200640 1 -- 172.21.15.103:0/3122209592 --> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f03e0104020 con 0x7f03c4039200 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.618+0000 7f03d6a00640 1 -- 172.21.15.103:0/3122209592 <== mgr.14118 v2:172.21.15.103:6800/623307529 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f03e0104020 con 0x7f03c4039200 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.620+0000 7f03df200640 1 -- 172.21.15.103:0/3122209592 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f03c4039200 msgr2=0x7f03c403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.620+0000 7f03df200640 1 --2- 172.21.15.103:0/3122209592 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f03c4039200 0x7f03c403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0098c0 tx=0x7f03cc006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.620+0000 7f03df200640 1 -- 172.21.15.103:0/3122209592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 msgr2=0x7f03e019d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.620+0000 7f03df200640 1 --2- 172.21.15.103:0/3122209592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e019d810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009940 tx=0x7f03d0032ce0 comp rx=0 tx=0).stop 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.620+0000 7f03df200640 1 -- 172.21.15.103:0/3122209592 shutdown_connections 2024-09-16T01:56:09.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.620+0000 7f03df200640 1 --2- 172.21.15.103:0/3122209592 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f03c4039200 0x7f03c403b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:09.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.620+0000 7f03df200640 1 --2- 172.21.15.103:0/3122209592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0107630 0x7f03e019d810 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:09.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.620+0000 7f03df200640 1 -- 172.21.15.103:0/3122209592 >> 172.21.15.103:0/3122209592 conn(0x7f03e0102dc0 msgr2=0x7f03e0103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:09.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.621+0000 7f03df200640 1 -- 172.21.15.103:0/3122209592 shutdown_connections 2024-09-16T01:56:09.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:08.621+0000 7f03df200640 1 -- 172.21.15.103:0/3122209592 wait complete. 2024-09-16T01:56:09.244 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T01:56:09.244 INFO:teuthology.orchestra.run.smithi103.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T01:56:09.258 INFO:teuthology.orchestra.run.smithi103.stdout:Adding host smithi103... 2024-09-16T01:56:09.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:09 smithi103 ceph-mon[30398]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:10.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:10 smithi103 ceph-mon[30398]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:11.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:11 smithi103 ceph-mon[30398]: Deploying cephadm binary to smithi103 2024-09-16T01:56:14.439 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Added host 'smithi103' with addr '172.21.15.103' 2024-09-16T01:56:14.439 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.781+0000 7f9844e00640 1 Processor -- start 2024-09-16T01:56:14.439 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.781+0000 7f9844e00640 1 -- start start 2024-09-16T01:56:14.439 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.781+0000 7f9844e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f9840109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:14.439 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.781+0000 7f9844e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f984010a240 con 0x7f9840109870 2024-09-16T01:56:14.439 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.782+0000 7f983f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f9840109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:14.439 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.782+0000 7f983f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f9840109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41112/0 (socket says 172.21.15.103:41112) 2024-09-16T01:56:14.439 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.782+0000 7f983f400640 1 -- 172.21.15.103:0/2986535909 learned_addr learned my addr 172.21.15.103:0/2986535909 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:14.439 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.782+0000 7f983f400640 1 -- 172.21.15.103:0/2986535909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984010aa10 con 0x7f9840109870 2024-09-16T01:56:14.439 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.783+0000 7f983f400640 1 --2- 172.21.15.103:0/2986535909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f9840109c70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9830009820 tx=0x7f983002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d45c7dc0b72af74 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.783+0000 7f983ea00640 1 -- 172.21.15.103:0/2986535909 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830037070 con 0x7f9840109870 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.783+0000 7f983ea00640 1 -- 172.21.15.103:0/2986535909 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9830032900 con 0x7f9840109870 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.783+0000 7f983ea00640 1 -- 172.21.15.103:0/2986535909 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830030850 con 0x7f9840109870 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.784+0000 7f9844e00640 1 -- 172.21.15.103:0/2986535909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 msgr2=0x7f9840109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.784+0000 7f9844e00640 1 --2- 172.21.15.103:0/2986535909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f9840109c70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9830009820 tx=0x7f983002a270 comp rx=0 tx=0).stop 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.784+0000 7f9844e00640 1 -- 172.21.15.103:0/2986535909 shutdown_connections 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.784+0000 7f9844e00640 1 --2- 172.21.15.103:0/2986535909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f9840109c70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.784+0000 7f9844e00640 1 -- 172.21.15.103:0/2986535909 >> 172.21.15.103:0/2986535909 conn(0x7f984007ce30 msgr2=0x7f984007d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.784+0000 7f9844e00640 1 -- 172.21.15.103:0/2986535909 shutdown_connections 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.784+0000 7f9844e00640 1 -- 172.21.15.103:0/2986535909 wait complete. 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.785+0000 7f9844e00640 1 Processor -- start 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.785+0000 7f9844e00640 1 -- start start 2024-09-16T01:56:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.785+0000 7f9844e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f984019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:14.441 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.785+0000 7f9844e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f984010ddc0 con 0x7f9840109870 2024-09-16T01:56:14.441 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.786+0000 7f983f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f984019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:14.441 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.786+0000 7f983f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f984019d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41122/0 (socket says 172.21.15.103:41122) 2024-09-16T01:56:14.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.786+0000 7f983f400640 1 -- 172.21.15.103:0/2552299194 learned_addr learned my addr 172.21.15.103:0/2552299194 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:14.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.786+0000 7f983f400640 1 -- 172.21.15.103:0/2552299194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984019dd70 con 0x7f9840109870 2024-09-16T01:56:14.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.786+0000 7f983f400640 1 --2- 172.21.15.103:0/2552299194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f984019d830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f98300097f0 tx=0x7f9830030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:14.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.786+0000 7f983cc00640 1 -- 172.21.15.103:0/2552299194 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830037040 con 0x7f9840109870 2024-09-16T01:56:14.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.787+0000 7f983cc00640 1 -- 172.21.15.103:0/2552299194 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9830040070 con 0x7f9840109870 2024-09-16T01:56:14.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.787+0000 7f9844e00640 1 -- 172.21.15.103:0/2552299194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984019e000 con 0x7f9840109870 2024-09-16T01:56:14.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.787+0000 7f983cc00640 1 -- 172.21.15.103:0/2552299194 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f983003a390 con 0x7f9840109870 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.787+0000 7f9844e00640 1 -- 172.21.15.103:0/2552299194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98401a0cf0 con 0x7f9840109870 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.788+0000 7f9844e00640 1 -- 172.21.15.103:0/2552299194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9840109cf0 con 0x7f9840109870 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.788+0000 7f983cc00640 1 -- 172.21.15.103:0/2552299194 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f983003b430 con 0x7f9840109870 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.788+0000 7f983cc00640 1 --2- 172.21.15.103:0/2552299194 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f98240391b0 0x7f982403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.788+0000 7f983cc00640 1 -- 172.21.15.103:0/2552299194 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f983006c860 con 0x7f9840109870 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.789+0000 7f9836a00640 1 --2- 172.21.15.103:0/2552299194 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f98240391b0 0x7f982403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.789+0000 7f9836a00640 1 --2- 172.21.15.103:0/2552299194 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f98240391b0 0x7f982403b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f98280098c0 tx=0x7f9828006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.793+0000 7f983cc00640 1 -- 172.21.15.103:0/2552299194 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f983006c1b0 con 0x7f9840109870 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:09.975+0000 7f9844e00640 1 -- 172.21.15.103:0/2552299194 --> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}) -- 0x7f9840107c10 con 0x7f98240391b0 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.758+0000 7f983cc00640 1 -- 172.21.15.103:0/2552299194 <== mgr.14118 v2:172.21.15.103:6800/623307529 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9840107c10 con 0x7f98240391b0 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.761+0000 7f9844e00640 1 -- 172.21.15.103:0/2552299194 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f98240391b0 msgr2=0x7f982403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:14.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.761+0000 7f9844e00640 1 --2- 172.21.15.103:0/2552299194 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f98240391b0 0x7f982403b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f98280098c0 tx=0x7f9828006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.761+0000 7f9844e00640 1 -- 172.21.15.103:0/2552299194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 msgr2=0x7f984019d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:14.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.761+0000 7f9844e00640 1 --2- 172.21.15.103:0/2552299194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f984019d830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f98300097f0 tx=0x7f9830030d80 comp rx=0 tx=0).stop 2024-09-16T01:56:14.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.761+0000 7f9844e00640 1 -- 172.21.15.103:0/2552299194 shutdown_connections 2024-09-16T01:56:14.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.761+0000 7f9844e00640 1 --2- 172.21.15.103:0/2552299194 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f98240391b0 0x7f982403b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.761+0000 7f9844e00640 1 --2- 172.21.15.103:0/2552299194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9840109870 0x7f984019d830 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.761+0000 7f9844e00640 1 -- 172.21.15.103:0/2552299194 >> 172.21.15.103:0/2552299194 conn(0x7f984007ce30 msgr2=0x7f9840107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:14.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.762+0000 7f9844e00640 1 -- 172.21.15.103:0/2552299194 shutdown_connections 2024-09-16T01:56:14.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:13.762+0000 7f9844e00640 1 -- 172.21.15.103:0/2552299194 wait complete. 2024-09-16T01:56:14.446 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mon service... 2024-09-16T01:56:14.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:14 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:14.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:14 smithi103 ceph-mon[30398]: Added host smithi103 2024-09-16T01:56:14.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:14 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:15.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T01:56:15.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.931+0000 7f5903e00640 1 Processor -- start 2024-09-16T01:56:15.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.931+0000 7f5903e00640 1 -- start start 2024-09-16T01:56:15.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.931+0000 7f5903e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f5904109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:15.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.931+0000 7f5903e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f590410a240 con 0x7f5904109870 2024-09-16T01:56:15.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.932+0000 7f5902a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f5904109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:15.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.932+0000 7f5902a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f5904109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41138/0 (socket says 172.21.15.103:41138) 2024-09-16T01:56:15.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.932+0000 7f5902a00640 1 -- 172.21.15.103:0/3403898949 learned_addr learned my addr 172.21.15.103:0/3403898949 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.932+0000 7f5902a00640 1 -- 172.21.15.103:0/3403898949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590410aa10 con 0x7f5904109870 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.933+0000 7f5902a00640 1 --2- 172.21.15.103:0/3403898949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f5904109c70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009820 tx=0x7f58f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54a467be98c9afe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.933+0000 7f5901600640 1 -- 172.21.15.103:0/3403898949 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f4037070 con 0x7f5904109870 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.933+0000 7f5901600640 1 -- 172.21.15.103:0/3403898949 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58f4032900 con 0x7f5904109870 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.933+0000 7f5901600640 1 -- 172.21.15.103:0/3403898949 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f4030850 con 0x7f5904109870 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.934+0000 7f5903e00640 1 -- 172.21.15.103:0/3403898949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 msgr2=0x7f5904109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.934+0000 7f5903e00640 1 --2- 172.21.15.103:0/3403898949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f5904109c70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009820 tx=0x7f58f402a270 comp rx=0 tx=0).stop 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.934+0000 7f5903e00640 1 -- 172.21.15.103:0/3403898949 shutdown_connections 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.934+0000 7f5903e00640 1 --2- 172.21.15.103:0/3403898949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f5904109c70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.934+0000 7f5903e00640 1 -- 172.21.15.103:0/3403898949 >> 172.21.15.103:0/3403898949 conn(0x7f590407ce30 msgr2=0x7f590407d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.934+0000 7f5903e00640 1 -- 172.21.15.103:0/3403898949 shutdown_connections 2024-09-16T01:56:15.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.934+0000 7f5903e00640 1 -- 172.21.15.103:0/3403898949 wait complete. 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.935+0000 7f5903e00640 1 Processor -- start 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.935+0000 7f5903e00640 1 -- start start 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.935+0000 7f5903e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f590419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.935+0000 7f5903e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f590410ddc0 con 0x7f5904109870 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.935+0000 7f5902a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f590419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.936+0000 7f5902a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f590419d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41146/0 (socket says 172.21.15.103:41146) 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.936+0000 7f5902a00640 1 -- 172.21.15.103:0/3746070789 learned_addr learned my addr 172.21.15.103:0/3746070789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.936+0000 7f5902a00640 1 -- 172.21.15.103:0/3746070789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590419dd70 con 0x7f5904109870 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.936+0000 7f5902a00640 1 --2- 172.21.15.103:0/3746070789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f590419d830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58f40097f0 tx=0x7f58f4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.936+0000 7f58fb400640 1 -- 172.21.15.103:0/3746070789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f4037040 con 0x7f5904109870 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.936+0000 7f58fb400640 1 -- 172.21.15.103:0/3746070789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58f4040070 con 0x7f5904109870 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.936+0000 7f58fb400640 1 -- 172.21.15.103:0/3746070789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f403a390 con 0x7f5904109870 2024-09-16T01:56:15.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.936+0000 7f5903e00640 1 -- 172.21.15.103:0/3746070789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590419e000 con 0x7f5904109870 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.937+0000 7f5903e00640 1 -- 172.21.15.103:0/3746070789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59041a0cf0 con 0x7f5904109870 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.937+0000 7f58fb400640 1 -- 172.21.15.103:0/3746070789 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f58f403b430 con 0x7f5904109870 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.938+0000 7f5903e00640 1 -- 172.21.15.103:0/3746070789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5904109cf0 con 0x7f5904109870 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.938+0000 7f58fb400640 1 --2- 172.21.15.103:0/3746070789 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f58e8039200 0x7f58e803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.938+0000 7f58fb400640 1 -- 172.21.15.103:0/3746070789 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f58f406c7d0 con 0x7f5904109870 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.938+0000 7f5902000640 1 --2- 172.21.15.103:0/3746070789 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f58e8039200 0x7f58e803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.939+0000 7f5902000640 1 --2- 172.21.15.103:0/3746070789 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f58e8039200 0x7f58e803b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f58f00098c0 tx=0x7f58f0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:14.943+0000 7f58fb400640 1 -- 172.21.15.103:0/3746070789 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58f406c120 con 0x7f5904109870 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.134+0000 7f5903e00640 1 -- 172.21.15.103:0/3746070789 --> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5904107c10 con 0x7f58e8039200 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.139+0000 7f58fb400640 1 -- 172.21.15.103:0/3746070789 <== mgr.14118 v2:172.21.15.103:6800/623307529 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5904107c10 con 0x7f58e8039200 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.141+0000 7f5903e00640 1 -- 172.21.15.103:0/3746070789 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f58e8039200 msgr2=0x7f58e803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:15.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.141+0000 7f5903e00640 1 --2- 172.21.15.103:0/3746070789 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f58e8039200 0x7f58e803b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f58f00098c0 tx=0x7f58f0006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:15.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.141+0000 7f5903e00640 1 -- 172.21.15.103:0/3746070789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 msgr2=0x7f590419d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:15.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.141+0000 7f5903e00640 1 --2- 172.21.15.103:0/3746070789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f590419d830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58f40097f0 tx=0x7f58f4030d80 comp rx=0 tx=0).stop 2024-09-16T01:56:15.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.141+0000 7f5903e00640 1 -- 172.21.15.103:0/3746070789 shutdown_connections 2024-09-16T01:56:15.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.141+0000 7f5903e00640 1 --2- 172.21.15.103:0/3746070789 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f58e8039200 0x7f58e803b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:15.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.141+0000 7f5903e00640 1 --2- 172.21.15.103:0/3746070789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5904109870 0x7f590419d830 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:15.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.141+0000 7f5903e00640 1 -- 172.21.15.103:0/3746070789 >> 172.21.15.103:0/3746070789 conn(0x7f590407ce30 msgr2=0x7f5904107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:15.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.142+0000 7f5903e00640 1 -- 172.21.15.103:0/3746070789 shutdown_connections 2024-09-16T01:56:15.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:15.142+0000 7f5903e00640 1 -- 172.21.15.103:0/3746070789 wait complete. 2024-09-16T01:56:15.863 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mgr service... 2024-09-16T01:56:16.447 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:16 smithi103 ceph-mon[30398]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:16.447 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:16 smithi103 ceph-mon[30398]: Saving service mon spec with placement count:5 2024-09-16T01:56:16.447 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:16 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:17.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T01:56:17.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.911+0000 7f0a6f200640 1 Processor -- start 2024-09-16T01:56:17.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.911+0000 7f0a6f200640 1 -- start start 2024-09-16T01:56:17.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.912+0000 7f0a6f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6807c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.912+0000 7f0a6f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6807c7b0 con 0x7f0a6807de10 2024-09-16T01:56:17.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.912+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.912+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6807c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41150/0 (socket says 172.21.15.103:41150) 2024-09-16T01:56:17.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.912+0000 7f0a67e00640 1 -- 172.21.15.103:0/683171189 learned_addr learned my addr 172.21.15.103:0/683171189 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:17.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.913+0000 7f0a67e00640 1 -- 172.21.15.103:0/683171189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6807c930 con 0x7f0a6807de10 2024-09-16T01:56:17.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.914+0000 7f0a67e00640 1 --2- 172.21.15.103:0/683171189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6807c270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009820 tx=0x7f0a5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab50094d79fcdb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:17.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.914+0000 7f0a66a00640 1 -- 172.21.15.103:0/683171189 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a58037070 con 0x7f0a6807de10 2024-09-16T01:56:17.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.914+0000 7f0a66a00640 1 -- 172.21.15.103:0/683171189 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a58032900 con 0x7f0a6807de10 2024-09-16T01:56:17.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.914+0000 7f0a66a00640 1 -- 172.21.15.103:0/683171189 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a58030850 con 0x7f0a6807de10 2024-09-16T01:56:17.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.914+0000 7f0a6f200640 1 -- 172.21.15.103:0/683171189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 msgr2=0x7f0a6807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.914+0000 7f0a6f200640 1 --2- 172.21.15.103:0/683171189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6807c270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009820 tx=0x7f0a5802a270 comp rx=0 tx=0).stop 2024-09-16T01:56:17.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.915+0000 7f0a6f200640 1 -- 172.21.15.103:0/683171189 shutdown_connections 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.915+0000 7f0a6f200640 1 --2- 172.21.15.103:0/683171189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6807c270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.915+0000 7f0a6f200640 1 -- 172.21.15.103:0/683171189 >> 172.21.15.103:0/683171189 conn(0x7f0a68102ff0 msgr2=0x7f0a68105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.915+0000 7f0a6f200640 1 -- 172.21.15.103:0/683171189 shutdown_connections 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.915+0000 7f0a6f200640 1 -- 172.21.15.103:0/683171189 wait complete. 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.916+0000 7f0a6f200640 1 Processor -- start 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.916+0000 7f0a6f200640 1 -- start start 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.916+0000 7f0a6f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.916+0000 7f0a6f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6810de50 con 0x7f0a6807de10 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.916+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.916+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41160/0 (socket says 172.21.15.103:41160) 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.916+0000 7f0a67e00640 1 -- 172.21.15.103:0/3389289786 learned_addr learned my addr 172.21.15.103:0/3389289786 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:17.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.917+0000 7f0a67e00640 1 -- 172.21.15.103:0/3389289786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6819dfa0 con 0x7f0a6807de10 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.917+0000 7f0a67e00640 1 --2- 172.21.15.103:0/3389289786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6819da60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0a580097f0 tx=0x7f0a58030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.917+0000 7f0a64c00640 1 -- 172.21.15.103:0/3389289786 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a58042070 con 0x7f0a6807de10 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.917+0000 7f0a64c00640 1 -- 172.21.15.103:0/3389289786 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a58032ca0 con 0x7f0a6807de10 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.917+0000 7f0a6f200640 1 -- 172.21.15.103:0/3389289786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6819e230 con 0x7f0a6807de10 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.917+0000 7f0a64c00640 1 -- 172.21.15.103:0/3389289786 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a58037040 con 0x7f0a6807de10 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.917+0000 7f0a6f200640 1 -- 172.21.15.103:0/3389289786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a681a0f20 con 0x7f0a6807de10 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.918+0000 7f0a6f200640 1 -- 172.21.15.103:0/3389289786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6807c2f0 con 0x7f0a6807de10 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.918+0000 7f0a64c00640 1 -- 172.21.15.103:0/3389289786 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0a58045070 con 0x7f0a6807de10 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.919+0000 7f0a64c00640 1 --2- 172.21.15.103:0/3389289786 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f0a4c039200 0x7f0a4c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.919+0000 7f0a64c00640 1 -- 172.21.15.103:0/3389289786 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0a5806c2e0 con 0x7f0a6807de10 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.919+0000 7f0a67400640 1 --2- 172.21.15.103:0/3389289786 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f0a4c039200 0x7f0a4c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.920+0000 7f0a67400640 1 --2- 172.21.15.103:0/3389289786 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f0a4c039200 0x7f0a4c03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0a540098c0 tx=0x7f0a54006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:17.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:16.924+0000 7f0a64c00640 1 -- 172.21.15.103:0/3389289786 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a5806bc30 con 0x7f0a6807de10 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.115+0000 7f0a6f200640 1 -- 172.21.15.103:0/3389289786 --> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0a68104380 con 0x7f0a4c039200 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.120+0000 7f0a64c00640 1 -- 172.21.15.103:0/3389289786 <== mgr.14118 v2:172.21.15.103:6800/623307529 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0a68104380 con 0x7f0a4c039200 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.122+0000 7f0a6f200640 1 -- 172.21.15.103:0/3389289786 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f0a4c039200 msgr2=0x7f0a4c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.122+0000 7f0a6f200640 1 --2- 172.21.15.103:0/3389289786 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f0a4c039200 0x7f0a4c03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0a540098c0 tx=0x7f0a54006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.122+0000 7f0a6f200640 1 -- 172.21.15.103:0/3389289786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 msgr2=0x7f0a6819da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.122+0000 7f0a6f200640 1 --2- 172.21.15.103:0/3389289786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6819da60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0a580097f0 tx=0x7f0a58030ec0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.122+0000 7f0a6f200640 1 -- 172.21.15.103:0/3389289786 shutdown_connections 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.122+0000 7f0a6f200640 1 --2- 172.21.15.103:0/3389289786 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f0a4c039200 0x7f0a4c03b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.122+0000 7f0a6f200640 1 --2- 172.21.15.103:0/3389289786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a6807de10 0x7f0a6819da60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.122+0000 7f0a6f200640 1 -- 172.21.15.103:0/3389289786 >> 172.21.15.103:0/3389289786 conn(0x7f0a68102ff0 msgr2=0x7f0a68103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.123+0000 7f0a6f200640 1 -- 172.21.15.103:0/3389289786 shutdown_connections 2024-09-16T01:56:17.935 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:17.123+0000 7f0a6f200640 1 -- 172.21.15.103:0/3389289786 wait complete. 2024-09-16T01:56:18.182 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:18 smithi103 ceph-mon[30398]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:18.182 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:18 smithi103 ceph-mon[30398]: Saving service mgr spec with placement count:2 2024-09-16T01:56:18.182 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:18 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:18.182 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:18 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:18.182 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:18 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:19.151 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.389+0000 7fdbeea00640 1 Processor -- start 2024-09-16T01:56:19.152 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.389+0000 7fdbeea00640 1 -- start start 2024-09-16T01:56:19.152 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.389+0000 7fdbeea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8107c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:19.152 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.389+0000 7fdbeea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81081d0 con 0x7fdbe8107800 2024-09-16T01:56:19.152 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.390+0000 7fdbe7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8107c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:19.152 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.390+0000 7fdbe7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8107c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45362/0 (socket says 172.21.15.103:45362) 2024-09-16T01:56:19.152 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.390+0000 7fdbe7e00640 1 -- 172.21.15.103:0/568788886 learned_addr learned my addr 172.21.15.103:0/568788886 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:19.152 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.390+0000 7fdbe7e00640 1 -- 172.21.15.103:0/568788886 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe8108a00 con 0x7fdbe8107800 2024-09-16T01:56:19.152 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.391+0000 7fdbe7e00640 1 --2- 172.21.15.103:0/568788886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8107c00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009820 tx=0x7fdbd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65209ac6ead7db1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:19.152 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.392+0000 7fdbe6a00640 1 -- 172.21.15.103:0/568788886 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbd8037070 con 0x7fdbe8107800 2024-09-16T01:56:19.152 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.392+0000 7fdbe6a00640 1 -- 172.21.15.103:0/568788886 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd8032920 con 0x7fdbe8107800 2024-09-16T01:56:19.153 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.392+0000 7fdbe6a00640 1 -- 172.21.15.103:0/568788886 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbd8030810 con 0x7fdbe8107800 2024-09-16T01:56:19.153 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.392+0000 7fdbeea00640 1 -- 172.21.15.103:0/568788886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 msgr2=0x7fdbe8107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:19.153 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.392+0000 7fdbeea00640 1 --2- 172.21.15.103:0/568788886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8107c00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009820 tx=0x7fdbd802a270 comp rx=0 tx=0).stop 2024-09-16T01:56:19.153 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.392+0000 7fdbeea00640 1 -- 172.21.15.103:0/568788886 shutdown_connections 2024-09-16T01:56:19.153 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.392+0000 7fdbeea00640 1 --2- 172.21.15.103:0/568788886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8107c00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:19.153 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.392+0000 7fdbeea00640 1 -- 172.21.15.103:0/568788886 >> 172.21.15.103:0/568788886 conn(0x7fdbe8102ff0 msgr2=0x7fdbe8105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:19.153 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.392+0000 7fdbeea00640 1 -- 172.21.15.103:0/568788886 shutdown_connections 2024-09-16T01:56:19.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.392+0000 7fdbeea00640 1 -- 172.21.15.103:0/568788886 wait complete. 2024-09-16T01:56:19.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.393+0000 7fdbeea00640 1 Processor -- start 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.393+0000 7fdbeea00640 1 -- start start 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.394+0000 7fdbeea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.394+0000 7fdbeea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe807c210 con 0x7fdbe8107800 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.394+0000 7fdbe7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.394+0000 7fdbe7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45364/0 (socket says 172.21.15.103:45364) 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.394+0000 7fdbe7e00640 1 -- 172.21.15.103:0/1479524793 learned_addr learned my addr 172.21.15.103:0/1479524793 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.394+0000 7fdbe7e00640 1 -- 172.21.15.103:0/1479524793 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe8199bd0 con 0x7fdbe8107800 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.394+0000 7fdbe7e00640 1 --2- 172.21.15.103:0/1479524793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8199690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8030560 tx=0x7fdbd8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.395+0000 7fdbe4c00640 1 -- 172.21.15.103:0/1479524793 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbd8037040 con 0x7fdbe8107800 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.395+0000 7fdbe4c00640 1 -- 172.21.15.103:0/1479524793 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd8040070 con 0x7fdbe8107800 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.395+0000 7fdbeea00640 1 -- 172.21.15.103:0/1479524793 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe8199e60 con 0x7fdbe8107800 2024-09-16T01:56:19.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.395+0000 7fdbe4c00640 1 -- 172.21.15.103:0/1479524793 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbd803a6d0 con 0x7fdbe8107800 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.395+0000 7fdbeea00640 1 -- 172.21.15.103:0/1479524793 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe819cb50 con 0x7fdbe8107800 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.396+0000 7fdbe4c00640 1 -- 172.21.15.103:0/1479524793 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fdbd803bb20 con 0x7fdbe8107800 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.396+0000 7fdbeea00640 1 -- 172.21.15.103:0/1479524793 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe8107c00 con 0x7fdbe8107800 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.396+0000 7fdbe4c00640 1 --2- 172.21.15.103:0/1479524793 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fdbcc03d660 0x7fdbcc03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.396+0000 7fdbe4c00640 1 -- 172.21.15.103:0/1479524793 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdbd806c880 con 0x7fdbe8107800 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.397+0000 7fdbe7400640 1 --2- 172.21.15.103:0/1479524793 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fdbcc03d660 0x7fdbcc03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.397+0000 7fdbe7400640 1 --2- 172.21.15.103:0/1479524793 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fdbcc03d660 0x7fdbcc03fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40098c0 tx=0x7fdbd4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.402+0000 7fdbe4c00640 1 -- 172.21.15.103:0/1479524793 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdbd8043360 con 0x7fdbe8107800 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.590+0000 7fdbeea00640 1 -- 172.21.15.103:0/1479524793 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fdbe802d9b0 con 0x7fdbe8107800 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.592+0000 7fdbe4c00640 1 -- 172.21.15.103:0/1479524793 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fdbd8040220 con 0x7fdbe8107800 2024-09-16T01:56:19.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.595+0000 7fdbeea00640 1 -- 172.21.15.103:0/1479524793 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fdbcc03d660 msgr2=0x7fdbcc03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:19.159 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.595+0000 7fdbeea00640 1 --2- 172.21.15.103:0/1479524793 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fdbcc03d660 0x7fdbcc03fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40098c0 tx=0x7fdbd4006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:19.159 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.595+0000 7fdbeea00640 1 -- 172.21.15.103:0/1479524793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 msgr2=0x7fdbe8199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:19.159 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.595+0000 7fdbeea00640 1 --2- 172.21.15.103:0/1479524793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8199690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8030560 tx=0x7fdbd8032670 comp rx=0 tx=0).stop 2024-09-16T01:56:19.159 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.596+0000 7fdbeea00640 1 -- 172.21.15.103:0/1479524793 shutdown_connections 2024-09-16T01:56:19.159 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.596+0000 7fdbeea00640 1 --2- 172.21.15.103:0/1479524793 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fdbcc03d660 0x7fdbcc03fb20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:19.159 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.596+0000 7fdbeea00640 1 --2- 172.21.15.103:0/1479524793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbe8107800 0x7fdbe8199690 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:19.159 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.596+0000 7fdbeea00640 1 -- 172.21.15.103:0/1479524793 >> 172.21.15.103:0/1479524793 conn(0x7fdbe8102ff0 msgr2=0x7fdbe8104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:19.159 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.596+0000 7fdbeea00640 1 -- 172.21.15.103:0/1479524793 shutdown_connections 2024-09-16T01:56:19.159 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:18.596+0000 7fdbeea00640 1 -- 172.21.15.103:0/1479524793 wait complete. 2024-09-16T01:56:19.813 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:19 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1479524793' entity='client.admin' 2024-09-16T01:56:21.124 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.258+0000 7f87f6600640 1 Processor -- start 2024-09-16T01:56:21.124 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.258+0000 7f87f6600640 1 -- start start 2024-09-16T01:56:21.124 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.258+0000 7f87f6600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f0107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:21.124 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.258+0000 7f87f6600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0108000 con 0x7f87f0107630 2024-09-16T01:56:21.124 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.259+0000 7f87f5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f0107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:21.124 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.259+0000 7f87f5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f0107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45372/0 (socket says 172.21.15.103:45372) 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.259+0000 7f87f5200640 1 -- 172.21.15.103:0/1690555718 learned_addr learned my addr 172.21.15.103:0/1690555718 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.259+0000 7f87f5200640 1 -- 172.21.15.103:0/1690555718 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f0108830 con 0x7f87f0107630 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.260+0000 7f87f5200640 1 --2- 172.21.15.103:0/1690555718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f0107a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009820 tx=0x7f87e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470d3c364fa45916 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.260+0000 7f87ef400640 1 -- 172.21.15.103:0/1690555718 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87e4037070 con 0x7f87f0107630 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.260+0000 7f87ef400640 1 -- 172.21.15.103:0/1690555718 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e4032920 con 0x7f87f0107630 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.261+0000 7f87ef400640 1 -- 172.21.15.103:0/1690555718 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87e4030810 con 0x7f87f0107630 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.261+0000 7f87f6600640 1 -- 172.21.15.103:0/1690555718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 msgr2=0x7f87f0107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.261+0000 7f87f6600640 1 --2- 172.21.15.103:0/1690555718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f0107a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009820 tx=0x7f87e402a270 comp rx=0 tx=0).stop 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.261+0000 7f87f6600640 1 -- 172.21.15.103:0/1690555718 shutdown_connections 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.261+0000 7f87f6600640 1 --2- 172.21.15.103:0/1690555718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f0107a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.261+0000 7f87f6600640 1 -- 172.21.15.103:0/1690555718 >> 172.21.15.103:0/1690555718 conn(0x7f87f0102dc0 msgr2=0x7f87f0105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.262+0000 7f87f6600640 1 -- 172.21.15.103:0/1690555718 shutdown_connections 2024-09-16T01:56:21.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.262+0000 7f87f6600640 1 -- 172.21.15.103:0/1690555718 wait complete. 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.263+0000 7f87f6600640 1 Processor -- start 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.263+0000 7f87f6600640 1 -- start start 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.263+0000 7f87f6600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f007db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.263+0000 7f87f6600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f87f010bbe0 con 0x7f87f0107630 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.263+0000 7f87f5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f007db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.263+0000 7f87f5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f007db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45376/0 (socket says 172.21.15.103:45376) 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.263+0000 7f87f5200640 1 -- 172.21.15.103:0/3075180505 learned_addr learned my addr 172.21.15.103:0/3075180505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.264+0000 7f87f5200640 1 -- 172.21.15.103:0/3075180505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f007e060 con 0x7f87f0107630 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.264+0000 7f87f5200640 1 --2- 172.21.15.103:0/3075180505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f007db20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87e4030560 tx=0x7f87e4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.264+0000 7f87ed600640 1 -- 172.21.15.103:0/3075180505 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87e4037040 con 0x7f87f0107630 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.264+0000 7f87ed600640 1 -- 172.21.15.103:0/3075180505 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e4040070 con 0x7f87f0107630 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.264+0000 7f87f6600640 1 -- 172.21.15.103:0/3075180505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f007c250 con 0x7f87f0107630 2024-09-16T01:56:21.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.264+0000 7f87ed600640 1 -- 172.21.15.103:0/3075180505 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87e403a740 con 0x7f87f0107630 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.265+0000 7f87f6600640 1 -- 172.21.15.103:0/3075180505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f007c790 con 0x7f87f0107630 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.266+0000 7f87f6600640 1 -- 172.21.15.103:0/3075180505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f0107ab0 con 0x7f87f0107630 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.266+0000 7f87ed600640 1 -- 172.21.15.103:0/3075180505 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f87e403b420 con 0x7f87f0107630 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.266+0000 7f87ed600640 1 --2- 172.21.15.103:0/3075180505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f87d8039200 0x7f87d803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.266+0000 7f87ed600640 1 -- 172.21.15.103:0/3075180505 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f87e406c8a0 con 0x7f87f0107630 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.267+0000 7f87efe00640 1 --2- 172.21.15.103:0/3075180505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f87d8039200 0x7f87d803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.267+0000 7f87efe00640 1 --2- 172.21.15.103:0/3075180505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f87d8039200 0x7f87d803b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0098c0 tx=0x7f87dc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.272+0000 7f87ed600640 1 -- 172.21.15.103:0/3075180505 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87e404a200 con 0x7f87f0107630 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.449+0000 7f87f6600640 1 -- 172.21.15.103:0/3075180505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f87f007d0e0 con 0x7f87f0107630 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.453+0000 7f87ed600640 1 -- 172.21.15.103:0/3075180505 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f87f007d0e0 con 0x7f87f0107630 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.456+0000 7f87f6600640 1 -- 172.21.15.103:0/3075180505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f87d8039200 msgr2=0x7f87d803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:21.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.456+0000 7f87f6600640 1 --2- 172.21.15.103:0/3075180505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f87d8039200 0x7f87d803b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0098c0 tx=0x7f87dc006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:21.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.456+0000 7f87f6600640 1 -- 172.21.15.103:0/3075180505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 msgr2=0x7f87f007db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:21.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.456+0000 7f87f6600640 1 --2- 172.21.15.103:0/3075180505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f007db20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87e4030560 tx=0x7f87e4032670 comp rx=0 tx=0).stop 2024-09-16T01:56:21.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.456+0000 7f87f6600640 1 -- 172.21.15.103:0/3075180505 shutdown_connections 2024-09-16T01:56:21.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.456+0000 7f87f6600640 1 --2- 172.21.15.103:0/3075180505 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f87d8039200 0x7f87d803b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:21.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.456+0000 7f87f6600640 1 --2- 172.21.15.103:0/3075180505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87f0107630 0x7f87f007db20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:21.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.456+0000 7f87f6600640 1 -- 172.21.15.103:0/3075180505 >> 172.21.15.103:0/3075180505 conn(0x7f87f0102dc0 msgr2=0x7f87f0103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:21.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.456+0000 7f87f6600640 1 -- 172.21.15.103:0/3075180505 shutdown_connections 2024-09-16T01:56:21.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:20.456+0000 7f87f6600640 1 -- 172.21.15.103:0/3075180505 wait complete. 2024-09-16T01:56:21.128 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling the dashboard module... 2024-09-16T01:56:21.380 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:21 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:21.380 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:21 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3075180505' entity='client.admin' 2024-09-16T01:56:22.544 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:22 smithi103 ceph-mon[30398]: from='mgr.14118 172.21.15.103:0/2872364660' entity='mgr.x' 2024-09-16T01:56:22.544 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:22 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2710989442' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T01:56:22.545 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:22 smithi103 ceph-mon[30398]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:56:22.934 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:22 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: ignoring --setuser ceph since I am not root 2024-09-16T01:56:22.934 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:22 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: ignoring --setgroup ceph since I am not root 2024-09-16T01:56:22.934 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:22 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:22.700+0000 7fced301a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:56:22.935 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:22 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:22.769+0000 7fced301a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:56:23.207 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:22 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:22.953+0000 7fced301a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:56:23.207 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:23 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:23.034+0000 7fced301a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:56:23.207 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:23 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:23.205+0000 7fced301a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:56:23.504 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:23 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:23.371+0000 7fced301a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:56:23.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:23 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2710989442' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T01:56:23.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:23 smithi103 ceph-mon[30398]: mgrmap e9: x(active, since 20s) 2024-09-16T01:56:23.667 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.639+0000 7f9f43200640 1 Processor -- start 2024-09-16T01:56:23.668 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.639+0000 7f9f43200640 1 -- start start 2024-09-16T01:56:23.668 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.640+0000 7f9f43200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c109e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.668 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.640+0000 7f9f43200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c10a3e0 con 0x7f9f3c109a10 2024-09-16T01:56:23.668 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.640+0000 7f9f3be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.668 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.640+0000 7f9f3be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c109e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45388/0 (socket says 172.21.15.103:45388) 2024-09-16T01:56:23.668 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.640+0000 7f9f3be00640 1 -- 172.21.15.103:0/1881028865 learned_addr learned my addr 172.21.15.103:0/1881028865 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:23.668 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.641+0000 7f9f3be00640 1 -- 172.21.15.103:0/1881028865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c10abc0 con 0x7f9f3c109a10 2024-09-16T01:56:23.668 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.642+0000 7f9f3be00640 1 --2- 172.21.15.103:0/1881028865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c109e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009820 tx=0x7f9f2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fbf2b606089c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:23.668 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.642+0000 7f9f3aa00640 1 -- 172.21.15.103:0/1881028865 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c037070 con 0x7f9f3c109a10 2024-09-16T01:56:23.668 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.642+0000 7f9f3aa00640 1 -- 172.21.15.103:0/1881028865 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c032920 con 0x7f9f3c109a10 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.642+0000 7f9f3aa00640 1 -- 172.21.15.103:0/1881028865 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c030810 con 0x7f9f3c109a10 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.643+0000 7f9f43200640 1 -- 172.21.15.103:0/1881028865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 msgr2=0x7f9f3c109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.643+0000 7f9f43200640 1 --2- 172.21.15.103:0/1881028865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c109e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009820 tx=0x7f9f2c02a270 comp rx=0 tx=0).stop 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.643+0000 7f9f43200640 1 -- 172.21.15.103:0/1881028865 shutdown_connections 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.643+0000 7f9f43200640 1 --2- 172.21.15.103:0/1881028865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c109e10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.643+0000 7f9f43200640 1 -- 172.21.15.103:0/1881028865 >> 172.21.15.103:0/1881028865 conn(0x7f9f3c07d030 msgr2=0x7f9f3c07d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.643+0000 7f9f43200640 1 -- 172.21.15.103:0/1881028865 shutdown_connections 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.643+0000 7f9f43200640 1 -- 172.21.15.103:0/1881028865 wait complete. 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.644+0000 7f9f43200640 1 Processor -- start 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.644+0000 7f9f43200640 1 -- start start 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.644+0000 7f9f43200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.669 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.644+0000 7f9f43200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c10df70 con 0x7f9f3c109a10 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.645+0000 7f9f3be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.645+0000 7f9f3be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c19da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45392/0 (socket says 172.21.15.103:45392) 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.645+0000 7f9f3be00640 1 -- 172.21.15.103:0/2710989442 learned_addr learned my addr 172.21.15.103:0/2710989442 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.645+0000 7f9f3be00640 1 -- 172.21.15.103:0/2710989442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c19dfa0 con 0x7f9f3c109a10 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.645+0000 7f9f3be00640 1 --2- 172.21.15.103:0/2710989442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c19da60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0097f0 tx=0x7f9f2c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.646+0000 7f9f38c00640 1 -- 172.21.15.103:0/2710989442 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c037040 con 0x7f9f3c109a10 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.646+0000 7f9f38c00640 1 -- 172.21.15.103:0/2710989442 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c030c40 con 0x7f9f3c109a10 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.646+0000 7f9f43200640 1 -- 172.21.15.103:0/2710989442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c19e230 con 0x7f9f3c109a10 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.646+0000 7f9f38c00640 1 -- 172.21.15.103:0/2710989442 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c03a5b0 con 0x7f9f3c109a10 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.646+0000 7f9f43200640 1 -- 172.21.15.103:0/2710989442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c1a0f20 con 0x7f9f3c109a10 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.647+0000 7f9f38c00640 1 -- 172.21.15.103:0/2710989442 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9f2c03b3f0 con 0x7f9f3c109a10 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.647+0000 7f9f43200640 1 -- 172.21.15.103:0/2710989442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3c10e0f0 con 0x7f9f3c109a10 2024-09-16T01:56:23.670 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.647+0000 7f9f38c00640 1 --2- 172.21.15.103:0/2710989442 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f9f200391b0 0x7f9f2003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.647+0000 7f9f38c00640 1 -- 172.21.15.103:0/2710989442 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9f2c06c770 con 0x7f9f3c109a10 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.648+0000 7f9f3b400640 1 --2- 172.21.15.103:0/2710989442 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f9f200391b0 0x7f9f2003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.648+0000 7f9f3b400640 1 --2- 172.21.15.103:0/2710989442 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f9f200391b0 0x7f9f2003b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9f280098c0 tx=0x7f9f28006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.652+0000 7f9f38c00640 1 -- 172.21.15.103:0/2710989442 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f2c0313a0 con 0x7f9f3c109a10 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:21.881+0000 7f9f43200640 1 -- 172.21.15.103:0/2710989442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f9f3c1a10f0 con 0x7f9f3c109a10 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.453+0000 7f9f38c00640 1 -- 172.21.15.103:0/2710989442 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f9f2c06c0c0 con 0x7f9f3c109a10 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.455+0000 7f9f38c00640 1 -- 172.21.15.103:0/2710989442 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f9f2c041d10 con 0x7f9f3c109a10 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.456+0000 7f9f43200640 1 -- 172.21.15.103:0/2710989442 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f9f200391b0 msgr2=0x7f9f2003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.456+0000 7f9f43200640 1 --2- 172.21.15.103:0/2710989442 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f9f200391b0 0x7f9f2003b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9f280098c0 tx=0x7f9f28006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.456+0000 7f9f43200640 1 -- 172.21.15.103:0/2710989442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 msgr2=0x7f9f3c19da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.456+0000 7f9f43200640 1 --2- 172.21.15.103:0/2710989442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c19da60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0097f0 tx=0x7f9f2c0327b0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.457+0000 7f9f43200640 1 -- 172.21.15.103:0/2710989442 shutdown_connections 2024-09-16T01:56:23.671 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.457+0000 7f9f43200640 1 --2- 172.21.15.103:0/2710989442 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f9f200391b0 0x7f9f2003b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.672 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.457+0000 7f9f43200640 1 --2- 172.21.15.103:0/2710989442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3c109a10 0x7f9f3c19da60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.672 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.457+0000 7f9f43200640 1 -- 172.21.15.103:0/2710989442 >> 172.21.15.103:0/2710989442 conn(0x7f9f3c07d030 msgr2=0x7f9f3c107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:23.672 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.457+0000 7f9f43200640 1 -- 172.21.15.103:0/2710989442 shutdown_connections 2024-09-16T01:56:23.672 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:22.457+0000 7f9f43200640 1 -- 172.21.15.103:0/2710989442 wait complete. 2024-09-16T01:56:23.996 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:23 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:23.748+0000 7fced301a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:56:23.996 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:23 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:23.820+0000 7fced301a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:56:24.252 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:24 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:24.036+0000 7fced301a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:56:24.848 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:24 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3357859594' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T01:56:25.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:56:25.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T01:56:25.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.277+0000 7fb51fa00640 1 Processor -- start 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.277+0000 7fb51fa00640 1 -- start start 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.278+0000 7fb51fa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb518109e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.278+0000 7fb51fa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810a3d0 con 0x7fb518109a00 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.278+0000 7fb51d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb518109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.278+0000 7fb51d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb518109e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45432/0 (socket says 172.21.15.103:45432) 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.278+0000 7fb51d000640 1 -- 172.21.15.103:0/1933143001 learned_addr learned my addr 172.21.15.103:0/1933143001 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.278+0000 7fb51d000640 1 -- 172.21.15.103:0/1933143001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51810abf0 con 0x7fb518109a00 2024-09-16T01:56:25.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.279+0000 7fb51d000640 1 --2- 172.21.15.103:0/1933143001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb518109e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009820 tx=0x7fb50c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e94f26def1ebb98 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.280+0000 7fb517400640 1 -- 172.21.15.103:0/1933143001 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb50c037070 con 0x7fb518109a00 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.280+0000 7fb517400640 1 -- 172.21.15.103:0/1933143001 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb50c032920 con 0x7fb518109a00 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.280+0000 7fb517400640 1 -- 172.21.15.103:0/1933143001 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb50c030810 con 0x7fb518109a00 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.280+0000 7fb51fa00640 1 -- 172.21.15.103:0/1933143001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 msgr2=0x7fb518109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.280+0000 7fb51fa00640 1 --2- 172.21.15.103:0/1933143001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb518109e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009820 tx=0x7fb50c02a270 comp rx=0 tx=0).stop 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.280+0000 7fb51fa00640 1 -- 172.21.15.103:0/1933143001 shutdown_connections 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.280+0000 7fb51fa00640 1 --2- 172.21.15.103:0/1933143001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb518109e00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.280+0000 7fb51fa00640 1 -- 172.21.15.103:0/1933143001 >> 172.21.15.103:0/1933143001 conn(0x7fb51807d010 msgr2=0x7fb51807d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.281+0000 7fb51fa00640 1 -- 172.21.15.103:0/1933143001 shutdown_connections 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.281+0000 7fb51fa00640 1 -- 172.21.15.103:0/1933143001 wait complete. 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.282+0000 7fb51fa00640 1 Processor -- start 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.282+0000 7fb51fa00640 1 -- start start 2024-09-16T01:56:25.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.282+0000 7fb51fa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb51819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.283+0000 7fb51fa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810dfa0 con 0x7fb518109a00 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.283+0000 7fb51d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb51819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.283+0000 7fb51d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb51819da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45444/0 (socket says 172.21.15.103:45444) 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.283+0000 7fb51d000640 1 -- 172.21.15.103:0/3357859594 learned_addr learned my addr 172.21.15.103:0/3357859594 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.283+0000 7fb51d000640 1 -- 172.21.15.103:0/3357859594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51819df60 con 0x7fb518109a00 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.283+0000 7fb51d000640 1 --2- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb51819da20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0097f0 tx=0x7fb50c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.284+0000 7fb515600640 1 -- 172.21.15.103:0/3357859594 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb50c037040 con 0x7fb518109a00 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.284+0000 7fb515600640 1 -- 172.21.15.103:0/3357859594 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb50c030cc0 con 0x7fb518109a00 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.284+0000 7fb51fa00640 1 -- 172.21.15.103:0/3357859594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51819e1f0 con 0x7fb518109a00 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.284+0000 7fb515600640 1 -- 172.21.15.103:0/3357859594 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb50c03a640 con 0x7fb518109a00 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.284+0000 7fb51fa00640 1 -- 172.21.15.103:0/3357859594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181a0ee0 con 0x7fb518109a00 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.285+0000 7fb515600640 1 -- 172.21.15.103:0/3357859594 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fb50c03a8f0 con 0x7fb518109a00 2024-09-16T01:56:25.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.285+0000 7fb51fa00640 1 -- 172.21.15.103:0/3357859594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb51810e120 con 0x7fb518109a00 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.286+0000 7fb515600640 1 --2- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb500039260 0x7fb50003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.286+0000 7fb515600640 1 -- 172.21.15.103:0/3357859594 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb50c06c8c0 con 0x7fb518109a00 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.286+0000 7fb517e00640 1 -- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb500039260 msgr2=0x7fb50003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/623307529 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.286+0000 7fb517e00640 1 --2- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb500039260 0x7fb50003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.291+0000 7fb515600640 1 -- 172.21.15.103:0/3357859594 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb50c0328c0 con 0x7fb518109a00 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.487+0000 7fb517e00640 1 -- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb500039260 msgr2=0x7fb50003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/623307529 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.487+0000 7fb517e00640 1 --2- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb500039260 0x7fb50003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.520+0000 7fb51fa00640 1 -- 172.21.15.103:0/3357859594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb5181a10b0 con 0x7fb518109a00 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.521+0000 7fb515600640 1 -- 172.21.15.103:0/3357859594 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb50c031c00 con 0x7fb518109a00 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.523+0000 7fb51fa00640 1 -- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb500039260 msgr2=0x7fb50003b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.523+0000 7fb51fa00640 1 --2- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb500039260 0x7fb50003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:25.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.523+0000 7fb51fa00640 1 -- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 msgr2=0x7fb51819da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:25.181 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.523+0000 7fb51fa00640 1 --2- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb51819da20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0097f0 tx=0x7fb50c0327b0 comp rx=0 tx=0).stop 2024-09-16T01:56:25.181 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.523+0000 7fb51fa00640 1 -- 172.21.15.103:0/3357859594 shutdown_connections 2024-09-16T01:56:25.181 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.523+0000 7fb51fa00640 1 --2- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7fb500039260 0x7fb50003b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:25.181 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.523+0000 7fb51fa00640 1 --2- 172.21.15.103:0/3357859594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518109a00 0x7fb51819da20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:25.181 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.523+0000 7fb51fa00640 1 -- 172.21.15.103:0/3357859594 >> 172.21.15.103:0/3357859594 conn(0x7fb51807d010 msgr2=0x7fb518107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:25.181 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.523+0000 7fb51fa00640 1 -- 172.21.15.103:0/3357859594 shutdown_connections 2024-09-16T01:56:25.181 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:24.523+0000 7fb51fa00640 1 -- 172.21.15.103:0/3357859594 wait complete. 2024-09-16T01:56:25.181 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-09-16T01:56:25.181 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 9... 2024-09-16T01:56:25.416 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:25 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:25.163+0000 7fced301a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:56:25.416 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:25 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:25.253+0000 7fced301a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:56:26.485 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:26.184+0000 7fced301a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:56:26.485 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:26.321+0000 7fced301a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:56:26.485 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:26.482+0000 7fced301a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:56:26.754 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:26.554+0000 7fced301a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:56:26.754 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:26.725+0000 7fced301a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:56:27.049 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:26.811+0000 7fced301a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:56:27.049 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:26.882+0000 7fced301a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:56:27.317 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:27.063+0000 7fced301a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:56:27.317 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:27.139+0000 7fced301a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:56:27.317 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:27.218+0000 7fced301a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:56:27.317 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:27.315+0000 7fced301a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:56:27.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:27.386+0000 7fced301a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:56:27.756 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:27.455+0000 7fced301a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:56:28.754 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:28.356+0000 7fced301a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:56:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: Active manager daemon x restarted 2024-09-16T01:56:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: Activating manager daemon x 2024-09-16T01:56:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T01:56:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: mgrmap e10: x(active, starting, since 0.0041483s) 2024-09-16T01:56:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:56:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:56:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:56:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:56:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:56:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: Manager daemon x is now available 2024-09-16T01:56:28.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:29.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:56:29.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:56:29.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-mon[30398]: mgrmap e11: x(active, since 1.00755s) 2024-09-16T01:56:30.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:29.962+0000 7fce29a00640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:01:56:29] ENGINE Error in HTTPServer.serve 2024-09-16T01:56:30.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: Traceback (most recent call last): 2024-09-16T01:56:30.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:56:30.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: self._connections.run(self.expiration_interval) 2024-09-16T01:56:30.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:56:30.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: self._run(expiration_interval) 2024-09-16T01:56:30.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: s = self.context.wrap_socket( 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: return self.sslsocket_class._create( 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: self.do_handshake() 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: self._sslobj.do_handshake() 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:56:30.256 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:56:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:56:30.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:56:30.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T01:56:30.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T01:56:30.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:56:30.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.635+0000 7f4d6ae00640 1 Processor -- start 2024-09-16T01:56:30.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.635+0000 7f4d6ae00640 1 -- start start 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.635+0000 7f4d6ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d64109e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.635+0000 7f4d6ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410a3d0 con 0x7f4d64109a00 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.636+0000 7f4d63e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d64109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.636+0000 7f4d63e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d64109e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45448/0 (socket says 172.21.15.103:45448) 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.636+0000 7f4d63e00640 1 -- 172.21.15.103:0/3946930576 learned_addr learned my addr 172.21.15.103:0/3946930576 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.636+0000 7f4d63e00640 1 -- 172.21.15.103:0/3946930576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6410ac00 con 0x7f4d64109a00 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.638+0000 7f4d63e00640 1 --2- 172.21.15.103:0/3946930576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d64109e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009820 tx=0x7f4d5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1164988b8125db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.638+0000 7f4d62a00640 1 -- 172.21.15.103:0/3946930576 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d54037070 con 0x7f4d64109a00 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.638+0000 7f4d62a00640 1 -- 172.21.15.103:0/3946930576 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d54032920 con 0x7f4d64109a00 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.638+0000 7f4d62a00640 1 -- 172.21.15.103:0/3946930576 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d54030810 con 0x7f4d64109a00 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.639+0000 7f4d6ae00640 1 -- 172.21.15.103:0/3946930576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 msgr2=0x7f4d64109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:30.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.639+0000 7f4d6ae00640 1 --2- 172.21.15.103:0/3946930576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d64109e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009820 tx=0x7f4d5402a270 comp rx=0 tx=0).stop 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.639+0000 7f4d6ae00640 1 -- 172.21.15.103:0/3946930576 shutdown_connections 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.639+0000 7f4d6ae00640 1 --2- 172.21.15.103:0/3946930576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d64109e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.639+0000 7f4d6ae00640 1 -- 172.21.15.103:0/3946930576 >> 172.21.15.103:0/3946930576 conn(0x7f4d6407cfe0 msgr2=0x7f4d6407d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.639+0000 7f4d6ae00640 1 -- 172.21.15.103:0/3946930576 shutdown_connections 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.639+0000 7f4d6ae00640 1 -- 172.21.15.103:0/3946930576 wait complete. 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.640+0000 7f4d6ae00640 1 Processor -- start 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.640+0000 7f4d6ae00640 1 -- start start 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.640+0000 7f4d6ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d6419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.640+0000 7f4d6ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410dfb0 con 0x7f4d64109a00 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.641+0000 7f4d63e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d6419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.641+0000 7f4d63e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d6419da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45454/0 (socket says 172.21.15.103:45454) 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.641+0000 7f4d63e00640 1 -- 172.21.15.103:0/1154244104 learned_addr learned my addr 172.21.15.103:0/1154244104 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:30.566 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.641+0000 7f4d63e00640 1 -- 172.21.15.103:0/1154244104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6419df60 con 0x7f4d64109a00 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.641+0000 7f4d63e00640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d6419da20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d540097f0 tx=0x7f4d540327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.642+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d54037040 con 0x7f4d64109a00 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.642+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d54030cc0 con 0x7f4d64109a00 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.642+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5403a640 con 0x7f4d64109a00 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.642+0000 7f4d6ae00640 1 -- 172.21.15.103:0/1154244104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6419e1f0 con 0x7f4d64109a00 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.642+0000 7f4d6ae00640 1 -- 172.21.15.103:0/1154244104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d641a0ee0 con 0x7f4d64109a00 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.643+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f4d5403b3f0 con 0x7f4d64109a00 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.643+0000 7f4d60c00640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 0x7f4d4803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.643+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 --> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4d4803be30 con 0x7f4d48039260 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.644+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4d5406c860 con 0x7f4d64109a00 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.644+0000 7f4d63400640 1 -- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 msgr2=0x7f4d4803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/623307529 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.644+0000 7f4d63400640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 0x7f4d4803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:56:30.567 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.845+0000 7f4d63400640 1 -- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 msgr2=0x7f4d4803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/623307529 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:25.845+0000 7f4d63400640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 0x7f4d4803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:26.246+0000 7f4d63400640 1 -- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 msgr2=0x7f4d4803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/623307529 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:26.246+0000 7f4d63400640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 0x7f4d4803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:27.047+0000 7f4d63400640 1 -- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 msgr2=0x7f4d4803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/623307529 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:27.047+0000 7f4d63400640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 0x7f4d4803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:28.361+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f4d54041d10 con 0x7f4d64109a00 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:28.361+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 msgr2=0x7f4d4803b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:28.361+0000 7f4d60c00640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 0x7f4d4803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.363+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f4d54042300 con 0x7f4d64109a00 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.363+0000 7f4d60c00640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d4803cd20 0x7f4d4803f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.363+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4d5406b100 con 0x7f4d4803cd20 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.364+0000 7f4d63400640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d4803cd20 0x7f4d4803f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.365+0000 7f4d63400640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d4803cd20 0x7f4d4803f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4d50003980 tx=0x7f4d500092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:30.568 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.366+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f4d5406b100 con 0x7f4d4803cd20 2024-09-16T01:56:30.569 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.373+0000 7f4d6ae00640 1 -- 172.21.15.103:0/1154244104 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4d6410e130 con 0x7f4d4803cd20 2024-09-16T01:56:30.569 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.373+0000 7f4d60c00640 1 -- 172.21.15.103:0/1154244104 <== mgr.14150 v2:172.21.15.103:6800/3075264030 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f4d6410e130 con 0x7f4d4803cd20 2024-09-16T01:56:30.569 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 -- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d4803cd20 msgr2=0x7f4d4803f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:30.569 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d4803cd20 0x7f4d4803f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4d50003980 tx=0x7f4d500092b0 comp rx=0 tx=0).stop 2024-09-16T01:56:30.569 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 -- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 msgr2=0x7f4d6419da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:30.569 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d6419da20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d540097f0 tx=0x7f4d540327b0 comp rx=0 tx=0).stop 2024-09-16T01:56:30.569 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 -- 172.21.15.103:0/1154244104 shutdown_connections 2024-09-16T01:56:30.569 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d4803cd20 0x7f4d4803f110 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:30.569 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:6800/623307529,v1:172.21.15.103:6801/623307529] conn(0x7f4d48039260 0x7f4d4803b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:30.569 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 --2- 172.21.15.103:0/1154244104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d64109a00 0x7f4d6419da20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:30.570 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 -- 172.21.15.103:0/1154244104 >> 172.21.15.103:0/1154244104 conn(0x7f4d6407cfe0 msgr2=0x7f4d64107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:30.570 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 -- 172.21.15.103:0/1154244104 shutdown_connections 2024-09-16T01:56:30.570 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:29.374+0000 7f4d6ae00640 1 -- 172.21.15.103:0/1154244104 wait complete. 2024-09-16T01:56:30.570 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 9 is available 2024-09-16T01:56:30.570 INFO:teuthology.orchestra.run.smithi103.stdout:Generating a dashboard self-signed certificate... 2024-09-16T01:56:30.707 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:30 smithi103 ceph-mon[30398]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T01:56:30.707 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:30 smithi103 ceph-mon[30398]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T01:56:30.707 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:31.687 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:31 smithi103 ceph-mon[30398]: [16/Sep/2024:01:56:29] ENGINE Bus STARTING 2024-09-16T01:56:31.687 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:31 smithi103 ceph-mon[30398]: [16/Sep/2024:01:56:29] ENGINE Serving on http://172.21.15.103:8765 2024-09-16T01:56:31.687 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:31 smithi103 ceph-mon[30398]: [16/Sep/2024:01:56:29] ENGINE Serving on https://172.21.15.103:7150 2024-09-16T01:56:31.687 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:31 smithi103 ceph-mon[30398]: [16/Sep/2024:01:56:29] ENGINE Bus STARTED 2024-09-16T01:56:31.687 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:31 smithi103 ceph-mon[30398]: [16/Sep/2024:01:56:29] ENGINE Error in HTTPServer.serve 2024-09-16T01:56:31.688 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-16T01:56:31.688 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:56:31.688 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-16T01:56:31.688 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:56:31.688 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-16T01:56:31.688 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:56:31.688 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:56:31.688 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:56:31.688 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:56:31.688 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:31 smithi103 ceph-mon[30398]: mgrmap e12: x(active, since 2s) 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:31 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:31.689 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:31 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:32.011 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T01:56:32.011 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.052+0000 7fdf5ca00640 1 Processor -- start 2024-09-16T01:56:32.011 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.052+0000 7fdf5ca00640 1 -- start start 2024-09-16T01:56:32.011 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.053+0000 7fdf5ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf58109e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:32.011 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.053+0000 7fdf5ca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5810a3e0 con 0x7fdf58109a10 2024-09-16T01:56:32.011 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.053+0000 7fdf55e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf58109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.053+0000 7fdf55e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf58109e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52236/0 (socket says 172.21.15.103:52236) 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.053+0000 7fdf55e00640 1 -- 172.21.15.103:0/3200094721 learned_addr learned my addr 172.21.15.103:0/3200094721 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.054+0000 7fdf55e00640 1 -- 172.21.15.103:0/3200094721 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5810abc0 con 0x7fdf58109a10 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.055+0000 7fdf55e00640 1 --2- 172.21.15.103:0/3200094721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf58109e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009e80 tx=0x7fdf4802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f93695ecf89dd91 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.055+0000 7fdf55400640 1 -- 172.21.15.103:0/3200094721 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf48036070 con 0x7fdf58109a10 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.055+0000 7fdf55400640 1 -- 172.21.15.103:0/3200094721 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf480318e0 con 0x7fdf58109a10 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.055+0000 7fdf55400640 1 -- 172.21.15.103:0/3200094721 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf48007a00 con 0x7fdf58109a10 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.055+0000 7fdf5ca00640 1 -- 172.21.15.103:0/3200094721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 msgr2=0x7fdf58109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.055+0000 7fdf5ca00640 1 --2- 172.21.15.103:0/3200094721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf58109e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009e80 tx=0x7fdf4802a7a0 comp rx=0 tx=0).stop 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.056+0000 7fdf5ca00640 1 -- 172.21.15.103:0/3200094721 shutdown_connections 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.056+0000 7fdf5ca00640 1 --2- 172.21.15.103:0/3200094721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf58109e10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.056+0000 7fdf5ca00640 1 -- 172.21.15.103:0/3200094721 >> 172.21.15.103:0/3200094721 conn(0x7fdf5807d030 msgr2=0x7fdf5807d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.056+0000 7fdf5ca00640 1 -- 172.21.15.103:0/3200094721 shutdown_connections 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.056+0000 7fdf5ca00640 1 -- 172.21.15.103:0/3200094721 wait complete. 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.057+0000 7fdf5ca00640 1 Processor -- start 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.057+0000 7fdf5ca00640 1 -- start start 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.057+0000 7fdf5ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf5819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.057+0000 7fdf5ca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5810df70 con 0x7fdf58109a10 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.057+0000 7fdf55e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf5819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.057+0000 7fdf55e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf5819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52250/0 (socket says 172.21.15.103:52250) 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.057+0000 7fdf55e00640 1 -- 172.21.15.103:0/2586900302 learned_addr learned my addr 172.21.15.103:0/2586900302 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.058+0000 7fdf55e00640 1 -- 172.21.15.103:0/2586900302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5819dfa0 con 0x7fdf58109a10 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.058+0000 7fdf55e00640 1 --2- 172.21.15.103:0/2586900302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf5819da60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf4802acc0 tx=0x7fdf48031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.058+0000 7fdf4f400640 1 -- 172.21.15.103:0/2586900302 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf48036050 con 0x7fdf58109a10 2024-09-16T01:56:32.013 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.058+0000 7fdf4f400640 1 -- 172.21.15.103:0/2586900302 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf48037040 con 0x7fdf58109a10 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.058+0000 7fdf4f400640 1 -- 172.21.15.103:0/2586900302 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf4803b920 con 0x7fdf58109a10 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.058+0000 7fdf5ca00640 1 -- 172.21.15.103:0/2586900302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5819e230 con 0x7fdf58109a10 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.059+0000 7fdf5ca00640 1 -- 172.21.15.103:0/2586900302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf581a0f20 con 0x7fdf58109a10 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.060+0000 7fdf5ca00640 1 -- 172.21.15.103:0/2586900302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5810e0f0 con 0x7fdf58109a10 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.060+0000 7fdf4f400640 1 -- 172.21.15.103:0/2586900302 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fdf48044020 con 0x7fdf58109a10 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.060+0000 7fdf4f400640 1 --2- 172.21.15.103:0/2586900302 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdf3c039140 0x7fdf3c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.060+0000 7fdf4f400640 1 -- 172.21.15.103:0/2586900302 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdf48042020 con 0x7fdf58109a10 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.061+0000 7fdf4d400640 1 --2- 172.21.15.103:0/2586900302 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdf3c039140 0x7fdf3c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.061+0000 7fdf4d400640 1 --2- 172.21.15.103:0/2586900302 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdf3c039140 0x7fdf3c03b600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdf400098c0 tx=0x7fdf40006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.065+0000 7fdf4f400640 1 -- 172.21.15.103:0/2586900302 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf4804b020 con 0x7fdf58109a10 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.255+0000 7fdf5ca00640 1 -- 172.21.15.103:0/2586900302 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fdf58107c80 con 0x7fdf3c039140 2024-09-16T01:56:32.014 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.285+0000 7fdf4f400640 1 -- 172.21.15.103:0/2586900302 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fdf4803d450 con 0x7fdf58109a10 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.347+0000 7fdf4f400640 1 -- 172.21.15.103:0/2586900302 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fdf58107c80 con 0x7fdf3c039140 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.349+0000 7fdf5ca00640 1 -- 172.21.15.103:0/2586900302 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdf3c039140 msgr2=0x7fdf3c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.349+0000 7fdf5ca00640 1 --2- 172.21.15.103:0/2586900302 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdf3c039140 0x7fdf3c03b600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdf400098c0 tx=0x7fdf40006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.349+0000 7fdf5ca00640 1 -- 172.21.15.103:0/2586900302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 msgr2=0x7fdf5819da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.349+0000 7fdf5ca00640 1 --2- 172.21.15.103:0/2586900302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf5819da60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf4802acc0 tx=0x7fdf48031cc0 comp rx=0 tx=0).stop 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.350+0000 7fdf5ca00640 1 -- 172.21.15.103:0/2586900302 shutdown_connections 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.350+0000 7fdf5ca00640 1 --2- 172.21.15.103:0/2586900302 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdf3c039140 0x7fdf3c03b600 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.350+0000 7fdf5ca00640 1 --2- 172.21.15.103:0/2586900302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf58109a10 0x7fdf5819da60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.350+0000 7fdf5ca00640 1 -- 172.21.15.103:0/2586900302 >> 172.21.15.103:0/2586900302 conn(0x7fdf5807d030 msgr2=0x7fdf58107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.350+0000 7fdf5ca00640 1 -- 172.21.15.103:0/2586900302 shutdown_connections 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:31.350+0000 7fdf5ca00640 1 -- 172.21.15.103:0/2586900302 wait complete. 2024-09-16T01:56:32.015 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial admin user... 2024-09-16T01:56:32.601 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:32 smithi103 ceph-mon[30398]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:32.602 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:33.703 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$C108KXEQWzKA8ZSA1FZEtudLkgI0x7OuMnfdUkaVw1hx3e3MKlyq.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726451792, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.471+0000 7fcf7be00640 1 Processor -- start 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.471+0000 7fcf7be00640 1 -- start start 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.472+0000 7fcf7be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c1062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.472+0000 7fcf7be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c07bde0 con 0x7fcf7c105ed0 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.472+0000 7fcf7aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c1062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.472+0000 7fcf7aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c1062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52258/0 (socket says 172.21.15.103:52258) 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.472+0000 7fcf7aa00640 1 -- 172.21.15.103:0/980237369 learned_addr learned my addr 172.21.15.103:0/980237369 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.473+0000 7fcf7aa00640 1 -- 172.21.15.103:0/980237369 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c106810 con 0x7fcf7c105ed0 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.474+0000 7fcf7aa00640 1 --2- 172.21.15.103:0/980237369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c1062d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009820 tx=0x7fcf6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7ddbc1b7d83a32 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.474+0000 7fcf79600640 1 -- 172.21.15.103:0/980237369 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf6c037070 con 0x7fcf7c105ed0 2024-09-16T01:56:33.704 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.474+0000 7fcf79600640 1 -- 172.21.15.103:0/980237369 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c032920 con 0x7fcf7c105ed0 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.474+0000 7fcf79600640 1 -- 172.21.15.103:0/980237369 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf6c030810 con 0x7fcf7c105ed0 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.475+0000 7fcf7be00640 1 -- 172.21.15.103:0/980237369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 msgr2=0x7fcf7c1062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.475+0000 7fcf7be00640 1 --2- 172.21.15.103:0/980237369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c1062d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009820 tx=0x7fcf6c02a270 comp rx=0 tx=0).stop 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.475+0000 7fcf7be00640 1 -- 172.21.15.103:0/980237369 shutdown_connections 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.475+0000 7fcf7be00640 1 --2- 172.21.15.103:0/980237369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c1062d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.475+0000 7fcf7be00640 1 -- 172.21.15.103:0/980237369 >> 172.21.15.103:0/980237369 conn(0x7fcf7c101ca0 msgr2=0x7fcf7c1040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.475+0000 7fcf7be00640 1 -- 172.21.15.103:0/980237369 shutdown_connections 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.475+0000 7fcf7be00640 1 -- 172.21.15.103:0/980237369 wait complete. 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.476+0000 7fcf7be00640 1 Processor -- start 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.476+0000 7fcf7be00640 1 -- start start 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.476+0000 7fcf7be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.476+0000 7fcf7be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c107950 con 0x7fcf7c105ed0 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.476+0000 7fcf7aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.705 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.476+0000 7fcf7aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c19d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52264/0 (socket says 172.21.15.103:52264) 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.476+0000 7fcf7aa00640 1 -- 172.21.15.103:0/1019632659 learned_addr learned my addr 172.21.15.103:0/1019632659 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.477+0000 7fcf7aa00640 1 -- 172.21.15.103:0/1019632659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c19dda0 con 0x7fcf7c105ed0 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.477+0000 7fcf7aa00640 1 --2- 172.21.15.103:0/1019632659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c19d860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0097f0 tx=0x7fcf6c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.477+0000 7fcf80a00640 1 -- 172.21.15.103:0/1019632659 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf6c037040 con 0x7fcf7c105ed0 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.477+0000 7fcf80a00640 1 -- 172.21.15.103:0/1019632659 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c030c40 con 0x7fcf7c105ed0 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.477+0000 7fcf7be00640 1 -- 172.21.15.103:0/1019632659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c19e030 con 0x7fcf7c105ed0 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.477+0000 7fcf80a00640 1 -- 172.21.15.103:0/1019632659 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf6c03a5d0 con 0x7fcf7c105ed0 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.478+0000 7fcf7be00640 1 -- 172.21.15.103:0/1019632659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c1a0d20 con 0x7fcf7c105ed0 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.479+0000 7fcf80a00640 1 -- 172.21.15.103:0/1019632659 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fcf6c03a810 con 0x7fcf7c105ed0 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.479+0000 7fcf7be00640 1 -- 172.21.15.103:0/1019632659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7c1062d0 con 0x7fcf7c105ed0 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.479+0000 7fcf80a00640 1 --2- 172.21.15.103:0/1019632659 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf60039200 0x7fcf6003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.479+0000 7fcf80a00640 1 -- 172.21.15.103:0/1019632659 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcf6c06c960 con 0x7fcf7c105ed0 2024-09-16T01:56:33.706 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.479+0000 7fcf7a000640 1 --2- 172.21.15.103:0/1019632659 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf60039200 0x7fcf6003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.480+0000 7fcf7a000640 1 --2- 172.21.15.103:0/1019632659 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf60039200 0x7fcf6003b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf680098c0 tx=0x7fcf68006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.484+0000 7fcf80a00640 1 -- 172.21.15.103:0/1019632659 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf6c0371e0 con 0x7fcf7c105ed0 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.681+0000 7fcf7be00640 1 -- 172.21.15.103:0/1019632659 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fcf7c19e510 con 0x7fcf60039200 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.910+0000 7fcf80a00640 1 -- 172.21.15.103:0/1019632659 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fcf7c19e510 con 0x7fcf60039200 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.912+0000 7fcf7be00640 1 -- 172.21.15.103:0/1019632659 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf60039200 msgr2=0x7fcf6003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.912+0000 7fcf7be00640 1 --2- 172.21.15.103:0/1019632659 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf60039200 0x7fcf6003b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf680098c0 tx=0x7fcf68006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.912+0000 7fcf7be00640 1 -- 172.21.15.103:0/1019632659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 msgr2=0x7fcf7c19d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.912+0000 7fcf7be00640 1 --2- 172.21.15.103:0/1019632659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c19d860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0097f0 tx=0x7fcf6c0327b0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.912+0000 7fcf7be00640 1 -- 172.21.15.103:0/1019632659 shutdown_connections 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.912+0000 7fcf7be00640 1 --2- 172.21.15.103:0/1019632659 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf60039200 0x7fcf6003b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.912+0000 7fcf7be00640 1 --2- 172.21.15.103:0/1019632659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf7c105ed0 0x7fcf7c19d860 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.912+0000 7fcf7be00640 1 -- 172.21.15.103:0/1019632659 >> 172.21.15.103:0/1019632659 conn(0x7fcf7c101ca0 msgr2=0x7fcf7c194520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.912+0000 7fcf7be00640 1 -- 172.21.15.103:0/1019632659 shutdown_connections 2024-09-16T01:56:33.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:32.912+0000 7fcf7be00640 1 -- 172.21.15.103:0/1019632659 wait complete. 2024-09-16T01:56:33.708 INFO:teuthology.orchestra.run.smithi103.stdout:Fetching dashboard port number... 2024-09-16T01:56:33.969 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:33 smithi103 ceph-mon[30398]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:33.969 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:33 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:35.104 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T01:56:35.104 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.312+0000 7fa363200640 1 Processor -- start 2024-09-16T01:56:35.104 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.312+0000 7fa363200640 1 -- start start 2024-09-16T01:56:35.104 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.312+0000 7fa363200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa364109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:35.104 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.312+0000 7fa363200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa36410a240 con 0x7fa364109870 2024-09-16T01:56:35.104 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.312+0000 7fa361e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa364109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:35.104 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.312+0000 7fa361e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa364109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52280/0 (socket says 172.21.15.103:52280) 2024-09-16T01:56:35.104 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.312+0000 7fa361e00640 1 -- 172.21.15.103:0/3831539072 learned_addr learned my addr 172.21.15.103:0/3831539072 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:35.104 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.313+0000 7fa361e00640 1 -- 172.21.15.103:0/3831539072 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36410aa10 con 0x7fa364109870 2024-09-16T01:56:35.104 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.314+0000 7fa361e00640 1 --2- 172.21.15.103:0/3831539072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa364109c70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa354009820 tx=0x7fa35402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dae757a3a95b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.314+0000 7fa361400640 1 -- 172.21.15.103:0/3831539072 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa354037070 con 0x7fa364109870 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.314+0000 7fa361400640 1 -- 172.21.15.103:0/3831539072 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa354032920 con 0x7fa364109870 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.314+0000 7fa361400640 1 -- 172.21.15.103:0/3831539072 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa354030810 con 0x7fa364109870 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.315+0000 7fa363200640 1 -- 172.21.15.103:0/3831539072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 msgr2=0x7fa364109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.315+0000 7fa363200640 1 --2- 172.21.15.103:0/3831539072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa364109c70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa354009820 tx=0x7fa35402a270 comp rx=0 tx=0).stop 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.315+0000 7fa363200640 1 -- 172.21.15.103:0/3831539072 shutdown_connections 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.315+0000 7fa363200640 1 --2- 172.21.15.103:0/3831539072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa364109c70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.315+0000 7fa363200640 1 -- 172.21.15.103:0/3831539072 >> 172.21.15.103:0/3831539072 conn(0x7fa36407ce30 msgr2=0x7fa36407d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.315+0000 7fa363200640 1 -- 172.21.15.103:0/3831539072 shutdown_connections 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.315+0000 7fa363200640 1 -- 172.21.15.103:0/3831539072 wait complete. 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.316+0000 7fa363200640 1 Processor -- start 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.316+0000 7fa363200640 1 -- start start 2024-09-16T01:56:35.105 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.316+0000 7fa363200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa36419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.316+0000 7fa363200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa36410ddc0 con 0x7fa364109870 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.316+0000 7fa361e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa36419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.316+0000 7fa361e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa36419d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52282/0 (socket says 172.21.15.103:52282) 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.316+0000 7fa361e00640 1 -- 172.21.15.103:0/2054231402 learned_addr learned my addr 172.21.15.103:0/2054231402 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.317+0000 7fa361e00640 1 -- 172.21.15.103:0/2054231402 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36419dd80 con 0x7fa364109870 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.317+0000 7fa361e00640 1 --2- 172.21.15.103:0/2054231402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa36419d840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa3540097f0 tx=0x7fa3540327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.317+0000 7fa35aa00640 1 -- 172.21.15.103:0/2054231402 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa354037040 con 0x7fa364109870 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.317+0000 7fa35aa00640 1 -- 172.21.15.103:0/2054231402 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa354030cc0 con 0x7fa364109870 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.317+0000 7fa363200640 1 -- 172.21.15.103:0/2054231402 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36419e010 con 0x7fa364109870 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.317+0000 7fa35aa00640 1 -- 172.21.15.103:0/2054231402 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa35403a640 con 0x7fa364109870 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.317+0000 7fa363200640 1 -- 172.21.15.103:0/2054231402 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3641a0d00 con 0x7fa364109870 2024-09-16T01:56:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.319+0000 7fa35aa00640 1 -- 172.21.15.103:0/2054231402 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fa35403b3f0 con 0x7fa364109870 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.319+0000 7fa363200640 1 -- 172.21.15.103:0/2054231402 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa36410df40 con 0x7fa364109870 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.319+0000 7fa35aa00640 1 --2- 172.21.15.103:0/2054231402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa348039260 0x7fa34803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.319+0000 7fa35aa00640 1 -- 172.21.15.103:0/2054231402 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa35406c950 con 0x7fa364109870 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.319+0000 7fa35be00640 1 --2- 172.21.15.103:0/2054231402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa348039260 0x7fa34803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.320+0000 7fa35be00640 1 --2- 172.21.15.103:0/2054231402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa348039260 0x7fa34803b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0098c0 tx=0x7fa34c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.324+0000 7fa35aa00640 1 -- 172.21.15.103:0/2054231402 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3540328c0 con 0x7fa364109870 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.506+0000 7fa363200640 1 -- 172.21.15.103:0/2054231402 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa3641a0ed0 con 0x7fa364109870 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.506+0000 7fa35aa00640 1 -- 172.21.15.103:0/2054231402 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa354042dc0 con 0x7fa364109870 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.509+0000 7fa363200640 1 -- 172.21.15.103:0/2054231402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa348039260 msgr2=0x7fa34803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.509+0000 7fa363200640 1 --2- 172.21.15.103:0/2054231402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa348039260 0x7fa34803b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0098c0 tx=0x7fa34c006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.509+0000 7fa363200640 1 -- 172.21.15.103:0/2054231402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 msgr2=0x7fa36419d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.509+0000 7fa363200640 1 --2- 172.21.15.103:0/2054231402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa36419d840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa3540097f0 tx=0x7fa3540327b0 comp rx=0 tx=0).stop 2024-09-16T01:56:35.107 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.509+0000 7fa363200640 1 -- 172.21.15.103:0/2054231402 shutdown_connections 2024-09-16T01:56:35.108 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.509+0000 7fa363200640 1 --2- 172.21.15.103:0/2054231402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa348039260 0x7fa34803b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:35.108 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.509+0000 7fa363200640 1 --2- 172.21.15.103:0/2054231402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa364109870 0x7fa36419d840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:35.108 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.509+0000 7fa363200640 1 -- 172.21.15.103:0/2054231402 >> 172.21.15.103:0/2054231402 conn(0x7fa36407ce30 msgr2=0x7fa364107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:35.108 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.509+0000 7fa363200640 1 -- 172.21.15.103:0/2054231402 shutdown_connections 2024-09-16T01:56:35.108 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:34.509+0000 7fa363200640 1 -- 172.21.15.103:0/2054231402 wait complete. 2024-09-16T01:56:35.120 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:56:35.120 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-16T01:56:35.133 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:56:35.134 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-16T01:56:35.134 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-16T01:56:35.134 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T01:56:35.136 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph Dashboard is now available at: 2024-09-16T01:56:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: URL: https://smithi103.front.sepia.ceph.com:8443/ 2024-09-16T01:56:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: User: admin 2024-09-16T01:56:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: Password: revypys5xk 2024-09-16T01:56:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:35.137 INFO:teuthology.orchestra.run.smithi103.stdout:Saving cluster configuration to /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config directory 2024-09-16T01:56:35.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:34 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2054231402' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T01:56:36.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:35 smithi103 ceph-mon[30398]: mgrmap e13: x(active, since 6s) 2024-09-16T01:56:36.652 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.734+0000 7f48e7800640 1 Processor -- start 2024-09-16T01:56:36.652 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.735+0000 7f48e7800640 1 -- start start 2024-09-16T01:56:36.652 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.735+0000 7f48e7800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e007c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.735+0000 7f48e7800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48e007c880 con 0x7f48e007de50 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.735+0000 7f48e4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e007c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.735+0000 7f48e4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e007c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52290/0 (socket says 172.21.15.103:52290) 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.735+0000 7f48e4e00640 1 -- 172.21.15.103:0/580209847 learned_addr learned my addr 172.21.15.103:0/580209847 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.736+0000 7f48e4e00640 1 -- 172.21.15.103:0/580209847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e007ca00 con 0x7f48e007de50 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.737+0000 7f48e4e00640 1 --2- 172.21.15.103:0/580209847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e007c2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009820 tx=0x7f48d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ae8b4649bac6d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.737+0000 7f48dfe00640 1 -- 172.21.15.103:0/580209847 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d0037070 con 0x7f48e007de50 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.737+0000 7f48dfe00640 1 -- 172.21.15.103:0/580209847 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d0032920 con 0x7f48e007de50 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.737+0000 7f48dfe00640 1 -- 172.21.15.103:0/580209847 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d0030810 con 0x7f48e007de50 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.737+0000 7f48e7800640 1 -- 172.21.15.103:0/580209847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 msgr2=0x7f48e007c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.737+0000 7f48e7800640 1 --2- 172.21.15.103:0/580209847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e007c2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009820 tx=0x7f48d002a270 comp rx=0 tx=0).stop 2024-09-16T01:56:36.653 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.738+0000 7f48e7800640 1 -- 172.21.15.103:0/580209847 shutdown_connections 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.738+0000 7f48e7800640 1 --2- 172.21.15.103:0/580209847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e007c2b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.738+0000 7f48e7800640 1 -- 172.21.15.103:0/580209847 >> 172.21.15.103:0/580209847 conn(0x7f48e0102ff0 msgr2=0x7f48e0105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.738+0000 7f48e7800640 1 -- 172.21.15.103:0/580209847 shutdown_connections 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.738+0000 7f48e7800640 1 -- 172.21.15.103:0/580209847 wait complete. 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.739+0000 7f48e7800640 1 Processor -- start 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.739+0000 7f48e7800640 1 -- start start 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.739+0000 7f48e7800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.739+0000 7f48e7800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48e010ded0 con 0x7f48e007de50 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.739+0000 7f48e4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.739+0000 7f48e4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e019da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52298/0 (socket says 172.21.15.103:52298) 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.739+0000 7f48e4e00640 1 -- 172.21.15.103:0/1135881627 learned_addr learned my addr 172.21.15.103:0/1135881627 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:36.654 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.740+0000 7f48e4e00640 1 -- 172.21.15.103:0/1135881627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e019dfa0 con 0x7f48e007de50 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.740+0000 7f48e4e00640 1 --2- 172.21.15.103:0/1135881627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e019da60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f48d00097f0 tx=0x7f48d00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.740+0000 7f48de000640 1 -- 172.21.15.103:0/1135881627 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d0037040 con 0x7f48e007de50 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.740+0000 7f48de000640 1 -- 172.21.15.103:0/1135881627 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d0030c40 con 0x7f48e007de50 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.740+0000 7f48de000640 1 -- 172.21.15.103:0/1135881627 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d003a5d0 con 0x7f48e007de50 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.740+0000 7f48e7800640 1 -- 172.21.15.103:0/1135881627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e019e230 con 0x7f48e007de50 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.740+0000 7f48e7800640 1 -- 172.21.15.103:0/1135881627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e01a0f20 con 0x7f48e007de50 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.741+0000 7f48de000640 1 -- 172.21.15.103:0/1135881627 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f48d003b3f0 con 0x7f48e007de50 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.741+0000 7f48e7800640 1 -- 172.21.15.103:0/1135881627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e007c330 con 0x7f48e007de50 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.742+0000 7f48de000640 1 --2- 172.21.15.103:0/1135881627 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48c4039300 0x7f48c403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.742+0000 7f48de000640 1 -- 172.21.15.103:0/1135881627 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48d006c9f0 con 0x7f48e007de50 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.742+0000 7f48d7e00640 1 --2- 172.21.15.103:0/1135881627 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48c4039300 0x7f48c403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.743+0000 7f48d7e00640 1 --2- 172.21.15.103:0/1135881627 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48c4039300 0x7f48c403b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f48c80098c0 tx=0x7f48c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:36.655 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:35.747+0000 7f48de000640 1 -- 172.21.15.103:0/1135881627 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48d00412b0 con 0x7f48e007de50 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.010+0000 7f48e7800640 1 -- 172.21.15.103:0/1135881627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f48e01a1510 con 0x7f48e007de50 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.012+0000 7f48de000640 1 -- 172.21.15.103:0/1135881627 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f48d003b6e0 con 0x7f48e007de50 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.015+0000 7f48e7800640 1 -- 172.21.15.103:0/1135881627 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48c4039300 msgr2=0x7f48c403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.015+0000 7f48e7800640 1 --2- 172.21.15.103:0/1135881627 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48c4039300 0x7f48c403b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f48c80098c0 tx=0x7f48c8006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.015+0000 7f48e7800640 1 -- 172.21.15.103:0/1135881627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 msgr2=0x7f48e019da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.015+0000 7f48e7800640 1 --2- 172.21.15.103:0/1135881627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e019da60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f48d00097f0 tx=0x7f48d00327b0 comp rx=0 tx=0).stop 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.015+0000 7f48e7800640 1 -- 172.21.15.103:0/1135881627 shutdown_connections 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.015+0000 7f48e7800640 1 --2- 172.21.15.103:0/1135881627 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48c4039300 0x7f48c403b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.015+0000 7f48e7800640 1 --2- 172.21.15.103:0/1135881627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e007de50 0x7f48e019da60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.015+0000 7f48e7800640 1 -- 172.21.15.103:0/1135881627 >> 172.21.15.103:0/1135881627 conn(0x7f48e0102ff0 msgr2=0x7f48e0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:36.656 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.015+0000 7f48e7800640 1 -- 172.21.15.103:0/1135881627 shutdown_connections 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-16T01:56:36.015+0000 7f48e7800640 1 -- 172.21.15.103:0/1135881627 wait complete. 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: ceph telemetry on 2024-09-16T01:56:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:36.658 INFO:teuthology.orchestra.run.smithi103.stdout:For more information see: 2024-09-16T01:56:36.658 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:36.658 INFO:teuthology.orchestra.run.smithi103.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T01:56:36.658 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:36.658 INFO:teuthology.orchestra.run.smithi103.stdout:Bootstrap complete. 2024-09-16T01:56:36.707 INFO:tasks.cephadm:Fetching config... 2024-09-16T01:56:36.707 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:56:36.708 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T01:56:36.745 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T01:56:36.745 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:56:36.745 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T01:56:36.822 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T01:56:36.822 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:56:36.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T01:56:36.917 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T01:56:36.918 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:56:36.918 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T01:56:36.993 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T01:56:36.993 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4qx/0LzvxSJ/kCPWrmyYgdq7IMssgpFGEAIacf9wKdAnlqkTR42Jw8uOPo9FVjlErBu5Ju2Gwz5EKlzPIRgpxMurgnV9HFxPp71Lj+p3KAkJr+SArtUIzPRbcnysSdNLuyoTFpzSKTzYTq7W8FoOvE+onZiGBev2RytTasvMEddQOerNjA7BzUs86amGC1Rc/KDq1s/VxFAQf4h9YPhPxhm5yTHDsGFeyofu+D+CmnQJtBK1aCfQujHgwz7IZaK08H5Ys+NLKlFr1vLkISPcz/Htw+XWbrEkOTzbmci2oFiiV1gWzad9xnG67TPRr+qK1D59KQtnEgnwRT74u0Su6kwa+JGYCdeHyeb8EcP5RaZKJtM3tFyiFMS+1p6roCMYqohN+hhiTYui3Nq6CrsiDlo8OlZTR28YP2gUk1qsnh0PjATInXVDFx7Ob95W96+9Af0TyjJUumJPA81BZq5gIp/lljAgBviTIpf0scqgubUJhkILsy1nYPsvHw7UXtp0= ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T01:56:37.123 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4qx/0LzvxSJ/kCPWrmyYgdq7IMssgpFGEAIacf9wKdAnlqkTR42Jw8uOPo9FVjlErBu5Ju2Gwz5EKlzPIRgpxMurgnV9HFxPp71Lj+p3KAkJr+SArtUIzPRbcnysSdNLuyoTFpzSKTzYTq7W8FoOvE+onZiGBev2RytTasvMEddQOerNjA7BzUs86amGC1Rc/KDq1s/VxFAQf4h9YPhPxhm5yTHDsGFeyofu+D+CmnQJtBK1aCfQujHgwz7IZaK08H5Ys+NLKlFr1vLkISPcz/Htw+XWbrEkOTzbmci2oFiiV1gWzad9xnG67TPRr+qK1D59KQtnEgnwRT74u0Su6kwa+JGYCdeHyeb8EcP5RaZKJtM3tFyiFMS+1p6roCMYqohN+hhiTYui3Nq6CrsiDlo8OlZTR28YP2gUk1qsnh0PjATInXVDFx7Ob95W96+9Af0TyjJUumJPA81BZq5gIp/lljAgBviTIpf0scqgubUJhkILsy1nYPsvHw7UXtp0= ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:56:37.148 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4qx/0LzvxSJ/kCPWrmyYgdq7IMssgpFGEAIacf9wKdAnlqkTR42Jw8uOPo9FVjlErBu5Ju2Gwz5EKlzPIRgpxMurgnV9HFxPp71Lj+p3KAkJr+SArtUIzPRbcnysSdNLuyoTFpzSKTzYTq7W8FoOvE+onZiGBev2RytTasvMEddQOerNjA7BzUs86amGC1Rc/KDq1s/VxFAQf4h9YPhPxhm5yTHDsGFeyofu+D+CmnQJtBK1aCfQujHgwz7IZaK08H5Ys+NLKlFr1vLkISPcz/Htw+XWbrEkOTzbmci2oFiiV1gWzad9xnG67TPRr+qK1D59KQtnEgnwRT74u0Su6kwa+JGYCdeHyeb8EcP5RaZKJtM3tFyiFMS+1p6roCMYqohN+hhiTYui3Nq6CrsiDlo8OlZTR28YP2gUk1qsnh0PjATInXVDFx7Ob95W96+9Af0TyjJUumJPA81BZq5gIp/lljAgBviTIpf0scqgubUJhkILsy1nYPsvHw7UXtp0= ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T01:56:37.190 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4qx/0LzvxSJ/kCPWrmyYgdq7IMssgpFGEAIacf9wKdAnlqkTR42Jw8uOPo9FVjlErBu5Ju2Gwz5EKlzPIRgpxMurgnV9HFxPp71Lj+p3KAkJr+SArtUIzPRbcnysSdNLuyoTFpzSKTzYTq7W8FoOvE+onZiGBev2RytTasvMEddQOerNjA7BzUs86amGC1Rc/KDq1s/VxFAQf4h9YPhPxhm5yTHDsGFeyofu+D+CmnQJtBK1aCfQujHgwz7IZaK08H5Ys+NLKlFr1vLkISPcz/Htw+XWbrEkOTzbmci2oFiiV1gWzad9xnG67TPRr+qK1D59KQtnEgnwRT74u0Su6kwa+JGYCdeHyeb8EcP5RaZKJtM3tFyiFMS+1p6roCMYqohN+hhiTYui3Nq6CrsiDlo8OlZTR28YP2gUk1qsnh0PjATInXVDFx7Ob95W96+9Af0TyjJUumJPA81BZq5gIp/lljAgBviTIpf0scqgubUJhkILsy1nYPsvHw7UXtp0= ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:56:37.203 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4qx/0LzvxSJ/kCPWrmyYgdq7IMssgpFGEAIacf9wKdAnlqkTR42Jw8uOPo9FVjlErBu5Ju2Gwz5EKlzPIRgpxMurgnV9HFxPp71Lj+p3KAkJr+SArtUIzPRbcnysSdNLuyoTFpzSKTzYTq7W8FoOvE+onZiGBev2RytTasvMEddQOerNjA7BzUs86amGC1Rc/KDq1s/VxFAQf4h9YPhPxhm5yTHDsGFeyofu+D+CmnQJtBK1aCfQujHgwz7IZaK08H5Ys+NLKlFr1vLkISPcz/Htw+XWbrEkOTzbmci2oFiiV1gWzad9xnG67TPRr+qK1D59KQtnEgnwRT74u0Su6kwa+JGYCdeHyeb8EcP5RaZKJtM3tFyiFMS+1p6roCMYqohN+hhiTYui3Nq6CrsiDlo8OlZTR28YP2gUk1qsnh0PjATInXVDFx7Ob95W96+9Af0TyjJUumJPA81BZq5gIp/lljAgBviTIpf0scqgubUJhkILsy1nYPsvHw7UXtp0= ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T01:56:37.212 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:37 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1135881627' entity='client.admin' 2024-09-16T01:56:37.245 INFO:teuthology.orchestra.run.smithi172.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4qx/0LzvxSJ/kCPWrmyYgdq7IMssgpFGEAIacf9wKdAnlqkTR42Jw8uOPo9FVjlErBu5Ju2Gwz5EKlzPIRgpxMurgnV9HFxPp71Lj+p3KAkJr+SArtUIzPRbcnysSdNLuyoTFpzSKTzYTq7W8FoOvE+onZiGBev2RytTasvMEddQOerNjA7BzUs86amGC1Rc/KDq1s/VxFAQf4h9YPhPxhm5yTHDsGFeyofu+D+CmnQJtBK1aCfQujHgwz7IZaK08H5Ys+NLKlFr1vLkISPcz/Htw+XWbrEkOTzbmci2oFiiV1gWzad9xnG67TPRr+qK1D59KQtnEgnwRT74u0Su6kwa+JGYCdeHyeb8EcP5RaZKJtM3tFyiFMS+1p6roCMYqohN+hhiTYui3Nq6CrsiDlo8OlZTR28YP2gUk1qsnh0PjATInXVDFx7Ob95W96+9Af0TyjJUumJPA81BZq5gIp/lljAgBviTIpf0scqgubUJhkILsy1nYPsvHw7UXtp0= ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:56:37.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T01:56:37.626 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.122+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1500909317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c20101400 msgr2=0x7f6c20101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.122+0000 7f6c1fe00640 1 --2- 172.21.15.103:0/1500909317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c20101400 0x7f6c20101800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009880 tx=0x7f6c1002a5d0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.123+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1500909317 shutdown_connections 2024-09-16T01:56:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.123+0000 7f6c1fe00640 1 --2- 172.21.15.103:0/1500909317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c20101400 0x7f6c20101800 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.123+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1500909317 >> 172.21.15.103:0/1500909317 conn(0x7f6c200fcb90 msgr2=0x7f6c200fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.123+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1500909317 shutdown_connections 2024-09-16T01:56:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.123+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1500909317 wait complete. 2024-09-16T01:56:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.124+0000 7f6c1fe00640 1 Processor -- start 2024-09-16T01:56:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.124+0000 7f6c1fe00640 1 -- start start 2024-09-16T01:56:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.124+0000 7f6c1fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c20101400 0x7f6c20190440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.124+0000 7f6c1fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c20107510 con 0x7f6c20101400 2024-09-16T01:56:38.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.125+0000 7f6c1ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c20101400 0x7f6c20190440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:38.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.125+0000 7f6c1ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c20101400 0x7f6c20190440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36836/0 (socket says 172.21.15.103:36836) 2024-09-16T01:56:38.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.125+0000 7f6c1ea00640 1 -- 172.21.15.103:0/1553129827 learned_addr learned my addr 172.21.15.103:0/1553129827 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:38.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.125+0000 7f6c1ea00640 1 -- 172.21.15.103:0/1553129827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c20190980 con 0x7f6c20101400 2024-09-16T01:56:38.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.125+0000 7f6c1ea00640 1 --2- 172.21.15.103:0/1553129827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c20101400 0x7f6c20190440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6c10004f60 tx=0x7f6c100048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.126+0000 7f6c17400640 1 -- 172.21.15.103:0/1553129827 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c1003f070 con 0x7f6c20101400 2024-09-16T01:56:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.126+0000 7f6c17400640 1 -- 172.21.15.103:0/1553129827 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c10004cc0 con 0x7f6c20101400 2024-09-16T01:56:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.126+0000 7f6c17400640 1 -- 172.21.15.103:0/1553129827 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c10035030 con 0x7f6c20101400 2024-09-16T01:56:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.126+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1553129827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c20190c10 con 0x7f6c20101400 2024-09-16T01:56:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.126+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1553129827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2018d1d0 con 0x7f6c20101400 2024-09-16T01:56:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.127+0000 7f6c17400640 1 -- 172.21.15.103:0/1553129827 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f6c10042070 con 0x7f6c20101400 2024-09-16T01:56:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.127+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1553129827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c20107c90 con 0x7f6c20101400 2024-09-16T01:56:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.127+0000 7f6c17400640 1 --2- 172.21.15.103:0/1553129827 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c04039300 0x7f6c0403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.128+0000 7f6c17400640 1 -- 172.21.15.103:0/1553129827 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c1006d0c0 con 0x7f6c20101400 2024-09-16T01:56:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.128+0000 7f6c1e000640 1 --2- 172.21.15.103:0/1553129827 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c04039300 0x7f6c0403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.129+0000 7f6c1e000640 1 --2- 172.21.15.103:0/1553129827 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c04039300 0x7f6c0403b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0098c0 tx=0x7f6c0c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:38.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.133+0000 7f6c17400640 1 -- 172.21.15.103:0/1553129827 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c10040260 con 0x7f6c20101400 2024-09-16T01:56:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.334+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1553129827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f6c201921d0 con 0x7f6c20101400 2024-09-16T01:56:38.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.337+0000 7f6c17400640 1 -- 172.21.15.103:0/1553129827 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f6c10003dd0 con 0x7f6c20101400 2024-09-16T01:56:38.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.339+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1553129827 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c04039300 msgr2=0x7f6c0403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:38.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.339+0000 7f6c1fe00640 1 --2- 172.21.15.103:0/1553129827 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c04039300 0x7f6c0403b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0098c0 tx=0x7f6c0c006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.339+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1553129827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c20101400 msgr2=0x7f6c20190440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:38.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.339+0000 7f6c1fe00640 1 --2- 172.21.15.103:0/1553129827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c20101400 0x7f6c20190440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6c10004f60 tx=0x7f6c100048f0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.339+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1553129827 shutdown_connections 2024-09-16T01:56:38.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.339+0000 7f6c1fe00640 1 --2- 172.21.15.103:0/1553129827 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c04039300 0x7f6c0403b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.339+0000 7f6c1fe00640 1 --2- 172.21.15.103:0/1553129827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c20101400 0x7f6c20190440 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.339+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1553129827 >> 172.21.15.103:0/1553129827 conn(0x7f6c200fcb90 msgr2=0x7f6c200fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:38.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.340+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1553129827 shutdown_connections 2024-09-16T01:56:38.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:38.340+0000 7f6c1fe00640 1 -- 172.21.15.103:0/1553129827 wait complete. 2024-09-16T01:56:39.007 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T01:56:39.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T01:56:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:39.606 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:39 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1553129827' entity='client.admin' 2024-09-16T01:56:40.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.118+0000 7fb302200640 1 -- 172.21.15.103:0/365010255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2fc1020b0 msgr2=0x7fb2fc1044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:40.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.118+0000 7fb302200640 1 --2- 172.21.15.103:0/365010255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2fc1020b0 0x7fb2fc1044a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009880 tx=0x7fb2ec02a5d0 comp rx=0 tx=0).stop 2024-09-16T01:56:40.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.118+0000 7fb302200640 1 -- 172.21.15.103:0/365010255 shutdown_connections 2024-09-16T01:56:40.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.118+0000 7fb302200640 1 --2- 172.21.15.103:0/365010255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2fc1020b0 0x7fb2fc1044a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:40.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.118+0000 7fb302200640 1 -- 172.21.15.103:0/365010255 >> 172.21.15.103:0/365010255 conn(0x7fb2fc0fbcb0 msgr2=0x7fb2fc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:40.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.119+0000 7fb302200640 1 -- 172.21.15.103:0/365010255 shutdown_connections 2024-09-16T01:56:40.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.119+0000 7fb302200640 1 -- 172.21.15.103:0/365010255 wait complete. 2024-09-16T01:56:40.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.120+0000 7fb302200640 1 Processor -- start 2024-09-16T01:56:40.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.120+0000 7fb302200640 1 -- start start 2024-09-16T01:56:40.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.120+0000 7fb302200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2fc1020b0 0x7fb2fc196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:40.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.120+0000 7fb302200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc1076c0 con 0x7fb2fc1020b0 2024-09-16T01:56:40.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.121+0000 7fb2fb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2fc1020b0 0x7fb2fc196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:40.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.121+0000 7fb2fb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2fc1020b0 0x7fb2fc196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36866/0 (socket says 172.21.15.103:36866) 2024-09-16T01:56:40.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.121+0000 7fb2fb400640 1 -- 172.21.15.103:0/834418293 learned_addr learned my addr 172.21.15.103:0/834418293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:40.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.121+0000 7fb2fb400640 1 -- 172.21.15.103:0/834418293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc197230 con 0x7fb2fc1020b0 2024-09-16T01:56:40.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.121+0000 7fb2fb400640 1 --2- 172.21.15.103:0/834418293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2fc1020b0 0x7fb2fc196cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec004f60 tx=0x7fb2ec0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:40.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.121+0000 7fb2f8c00640 1 -- 172.21.15.103:0/834418293 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2ec03f070 con 0x7fb2fc1020b0 2024-09-16T01:56:40.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.121+0000 7fb2f8c00640 1 -- 172.21.15.103:0/834418293 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2ec003ad0 con 0x7fb2fc1020b0 2024-09-16T01:56:40.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.121+0000 7fb302200640 1 -- 172.21.15.103:0/834418293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2fc197520 con 0x7fb2fc1020b0 2024-09-16T01:56:40.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.122+0000 7fb2f8c00640 1 -- 172.21.15.103:0/834418293 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2ec035030 con 0x7fb2fc1020b0 2024-09-16T01:56:40.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.122+0000 7fb302200640 1 -- 172.21.15.103:0/834418293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2fc19a1a0 con 0x7fb2fc1020b0 2024-09-16T01:56:40.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.123+0000 7fb2f8c00640 1 -- 172.21.15.103:0/834418293 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fb2ec042070 con 0x7fb2fc1020b0 2024-09-16T01:56:40.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.123+0000 7fb302200640 1 -- 172.21.15.103:0/834418293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2fc107e40 con 0x7fb2fc1020b0 2024-09-16T01:56:40.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.123+0000 7fb2f8c00640 1 --2- 172.21.15.103:0/834418293 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb2e00392a0 0x7fb2e003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:40.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.123+0000 7fb2f8c00640 1 -- 172.21.15.103:0/834418293 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb2ec06d0d0 con 0x7fb2fc1020b0 2024-09-16T01:56:40.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.123+0000 7fb2f2a00640 1 --2- 172.21.15.103:0/834418293 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb2e00392a0 0x7fb2e003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:40.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.124+0000 7fb2f2a00640 1 --2- 172.21.15.103:0/834418293 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb2e00392a0 0x7fb2e003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40098c0 tx=0x7fb2e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:40.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.128+0000 7fb2f8c00640 1 -- 172.21.15.103:0/834418293 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2ec036df0 con 0x7fb2fc1020b0 2024-09-16T01:56:40.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.324+0000 7fb302200640 1 -- 172.21.15.103:0/834418293 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb2fc02d940 con 0x7fb2e00392a0 2024-09-16T01:56:40.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.328+0000 7fb2f8c00640 1 -- 172.21.15.103:0/834418293 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb2fc02d940 con 0x7fb2e00392a0 2024-09-16T01:56:40.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.331+0000 7fb302200640 1 -- 172.21.15.103:0/834418293 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb2e00392a0 msgr2=0x7fb2e003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:40.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.331+0000 7fb302200640 1 --2- 172.21.15.103:0/834418293 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb2e00392a0 0x7fb2e003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40098c0 tx=0x7fb2e4006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:40.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.331+0000 7fb302200640 1 -- 172.21.15.103:0/834418293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2fc1020b0 msgr2=0x7fb2fc196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:40.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.331+0000 7fb302200640 1 --2- 172.21.15.103:0/834418293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2fc1020b0 0x7fb2fc196cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec004f60 tx=0x7fb2ec0048f0 comp rx=0 tx=0).stop 2024-09-16T01:56:40.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.331+0000 7fb302200640 1 -- 172.21.15.103:0/834418293 shutdown_connections 2024-09-16T01:56:40.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.331+0000 7fb302200640 1 --2- 172.21.15.103:0/834418293 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb2e00392a0 0x7fb2e003b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:40.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.331+0000 7fb302200640 1 --2- 172.21.15.103:0/834418293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2fc1020b0 0x7fb2fc196cf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:40.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.331+0000 7fb302200640 1 -- 172.21.15.103:0/834418293 >> 172.21.15.103:0/834418293 conn(0x7fb2fc0fbcb0 msgr2=0x7fb2fc0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:40.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.331+0000 7fb302200640 1 -- 172.21.15.103:0/834418293 shutdown_connections 2024-09-16T01:56:40.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:40.331+0000 7fb302200640 1 -- 172.21.15.103:0/834418293 wait complete. 2024-09-16T01:56:40.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:40.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:40.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:40.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:40.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:40.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:40.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:40.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:40.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:40.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:41.255 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-09-16T01:56:41.255 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:56:41.255 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-09-16T01:56:41.276 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:56:41.276 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:56:41.334 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-09-16T01:56:41.334 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch host add smithi136 2024-09-16T01:56:41.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:41 smithi103 ceph-mon[30398]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:41.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:41 smithi103 ceph-mon[30398]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-16T01:56:42.292 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:42.611 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:42 smithi103 ceph-mon[30398]: Updating smithi103:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:56:42.611 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:42 smithi103 ceph-mon[30398]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:56:42.611 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:42 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:42.611 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:42 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:42.611 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:42 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:42.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.866+0000 7f743e600640 1 -- 172.21.15.103:0/2463230701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7438076720 msgr2=0x7f7438076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:42.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.866+0000 7f743e600640 1 --2- 172.21.15.103:0/2463230701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7438076720 0x7f7438076b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7428009880 tx=0x7f742802a5d0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.866+0000 7f743e600640 1 -- 172.21.15.103:0/2463230701 shutdown_connections 2024-09-16T01:56:42.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.866+0000 7f743e600640 1 --2- 172.21.15.103:0/2463230701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7438076720 0x7f7438076b00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.866+0000 7f743e600640 1 -- 172.21.15.103:0/2463230701 >> 172.21.15.103:0/2463230701 conn(0x7f74380ff4e0 msgr2=0x7f7438101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:42.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.866+0000 7f743e600640 1 -- 172.21.15.103:0/2463230701 shutdown_connections 2024-09-16T01:56:42.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.866+0000 7f743e600640 1 -- 172.21.15.103:0/2463230701 wait complete. 2024-09-16T01:56:42.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.867+0000 7f743e600640 1 Processor -- start 2024-09-16T01:56:42.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.867+0000 7f743e600640 1 -- start start 2024-09-16T01:56:42.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.868+0000 7f743e600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7438076720 0x7f74381a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:42.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.868+0000 7f743e600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74381122d0 con 0x7f7438076720 2024-09-16T01:56:42.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.868+0000 7f7437400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7438076720 0x7f74381a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:42.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.868+0000 7f7437400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7438076720 0x7f74381a33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36880/0 (socket says 172.21.15.103:36880) 2024-09-16T01:56:42.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.868+0000 7f7437400640 1 -- 172.21.15.103:0/581142121 learned_addr learned my addr 172.21.15.103:0/581142121 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:42.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.868+0000 7f7437400640 1 -- 172.21.15.103:0/581142121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74381a3910 con 0x7f7438076720 2024-09-16T01:56:42.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.869+0000 7f7437400640 1 --2- 172.21.15.103:0/581142121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7438076720 0x7f74381a33d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7428009230 tx=0x7f7428003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:42.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.869+0000 7f7434c00640 1 -- 172.21.15.103:0/581142121 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f742803f070 con 0x7f7438076720 2024-09-16T01:56:42.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.869+0000 7f7434c00640 1 -- 172.21.15.103:0/581142121 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f742803bd40 con 0x7f7438076720 2024-09-16T01:56:42.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.869+0000 7f743e600640 1 -- 172.21.15.103:0/581142121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74381a3ba0 con 0x7f7438076720 2024-09-16T01:56:42.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.869+0000 7f7434c00640 1 -- 172.21.15.103:0/581142121 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7428035030 con 0x7f7438076720 2024-09-16T01:56:42.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.869+0000 7f743e600640 1 -- 172.21.15.103:0/581142121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743819d590 con 0x7f7438076720 2024-09-16T01:56:42.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.870+0000 7f743e600640 1 -- 172.21.15.103:0/581142121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7438077e60 con 0x7f7438076720 2024-09-16T01:56:42.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.870+0000 7f7434c00640 1 -- 172.21.15.103:0/581142121 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f7428042070 con 0x7f7438076720 2024-09-16T01:56:42.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.871+0000 7f7434c00640 1 --2- 172.21.15.103:0/581142121 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f741c039300 0x7f741c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:42.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.871+0000 7f7434c00640 1 -- 172.21.15.103:0/581142121 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f742806cfc0 con 0x7f7438076720 2024-09-16T01:56:42.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.871+0000 7f742ea00640 1 --2- 172.21.15.103:0/581142121 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f741c039300 0x7f741c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:42.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.872+0000 7f742ea00640 1 --2- 172.21.15.103:0/581142121 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f741c039300 0x7f741c03b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f74200098c0 tx=0x7f7420006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:42.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:42.875+0000 7f7434c00640 1 -- 172.21.15.103:0/581142121 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74280412c0 con 0x7f7438076720 2024-09-16T01:56:43.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:43.055+0000 7f743e600640 1 -- 172.21.15.103:0/581142121 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}) -- 0x7f743819eac0 con 0x7f741c039300 2024-09-16T01:56:43.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:43 smithi103 ceph-mon[30398]: Updating smithi103:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:56:44.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:44 smithi103 ceph-mon[30398]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:45.683 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:45 smithi103 ceph-mon[30398]: Deploying cephadm binary to smithi136 2024-09-16T01:56:46.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.176+0000 7f7434c00640 1 -- 172.21.15.103:0/581142121 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f743819eac0 con 0x7f741c039300 2024-09-16T01:56:46.178 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-09-16T01:56:46.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.179+0000 7f743e600640 1 -- 172.21.15.103:0/581142121 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f741c039300 msgr2=0x7f741c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:46.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.179+0000 7f743e600640 1 --2- 172.21.15.103:0/581142121 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f741c039300 0x7f741c03b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f74200098c0 tx=0x7f7420006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:46.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.179+0000 7f743e600640 1 -- 172.21.15.103:0/581142121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7438076720 msgr2=0x7f74381a33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:46.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.179+0000 7f743e600640 1 --2- 172.21.15.103:0/581142121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7438076720 0x7f74381a33d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7428009230 tx=0x7f7428003d80 comp rx=0 tx=0).stop 2024-09-16T01:56:46.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.179+0000 7f743e600640 1 -- 172.21.15.103:0/581142121 shutdown_connections 2024-09-16T01:56:46.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.179+0000 7f743e600640 1 --2- 172.21.15.103:0/581142121 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f741c039300 0x7f741c03b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:46.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.179+0000 7f743e600640 1 --2- 172.21.15.103:0/581142121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7438076720 0x7f74381a33d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:46.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.179+0000 7f743e600640 1 -- 172.21.15.103:0/581142121 >> 172.21.15.103:0/581142121 conn(0x7f74380ff4e0 msgr2=0x7f74380ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:46.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.180+0000 7f743e600640 1 -- 172.21.15.103:0/581142121 shutdown_connections 2024-09-16T01:56:46.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:46.180+0000 7f743e600640 1 -- 172.21.15.103:0/581142121 wait complete. 2024-09-16T01:56:46.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T01:56:47.250 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:47.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:47.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:47 smithi103 ceph-mon[30398]: Added host smithi136 2024-09-16T01:56:47.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:47.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:47.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:47.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.738+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2140606571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ed4101840 msgr2=0x7f0ed4101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.738+0000 7f0ed3e00640 1 --2- 172.21.15.103:0/2140606571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ed4101840 0x7f0ed4101c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009c20 tx=0x7f0ec402aa90 comp rx=0 tx=0).stop 2024-09-16T01:56:47.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.738+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2140606571 shutdown_connections 2024-09-16T01:56:47.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.738+0000 7f0ed3e00640 1 --2- 172.21.15.103:0/2140606571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ed4101840 0x7f0ed4101c20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.738+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2140606571 >> 172.21.15.103:0/2140606571 conn(0x7f0ed4078130 msgr2=0x7f0ed4078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:47.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.738+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2140606571 shutdown_connections 2024-09-16T01:56:47.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.738+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2140606571 wait complete. 2024-09-16T01:56:47.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.739+0000 7f0ed3e00640 1 Processor -- start 2024-09-16T01:56:47.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.739+0000 7f0ed3e00640 1 -- start start 2024-09-16T01:56:47.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.739+0000 7f0ed3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ed4101840 0x7f0ed41991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.739+0000 7f0ed3e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed410ba50 con 0x7f0ed4101840 2024-09-16T01:56:47.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.740+0000 7f0ed2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ed4101840 0x7f0ed41991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.740+0000 7f0ed2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ed4101840 0x7f0ed41991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36898/0 (socket says 172.21.15.103:36898) 2024-09-16T01:56:47.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.740+0000 7f0ed2a00640 1 -- 172.21.15.103:0/2613932240 learned_addr learned my addr 172.21.15.103:0/2613932240 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:47.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.740+0000 7f0ed2a00640 1 -- 172.21.15.103:0/2613932240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed4199720 con 0x7f0ed4101840 2024-09-16T01:56:47.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.740+0000 7f0ed2a00640 1 --2- 172.21.15.103:0/2613932240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ed4101840 0x7f0ed41991e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009d40 tx=0x7f0ec402fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:47.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.741+0000 7f0ed8a00640 1 -- 172.21.15.103:0/2613932240 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ec4047070 con 0x7f0ed4101840 2024-09-16T01:56:47.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.741+0000 7f0ed8a00640 1 -- 172.21.15.103:0/2613932240 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ec4003e40 con 0x7f0ed4101840 2024-09-16T01:56:47.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.741+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2613932240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed41999b0 con 0x7f0ed4101840 2024-09-16T01:56:47.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.741+0000 7f0ed8a00640 1 -- 172.21.15.103:0/2613932240 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ec4033050 con 0x7f0ed4101840 2024-09-16T01:56:47.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.741+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2613932240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed4199e10 con 0x7f0ed4101840 2024-09-16T01:56:47.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.742+0000 7f0ed8a00640 1 -- 172.21.15.103:0/2613932240 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f0ec404a070 con 0x7f0ed4101840 2024-09-16T01:56:47.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.742+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2613932240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed4102af0 con 0x7f0ed4101840 2024-09-16T01:56:47.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.742+0000 7f0ed8a00640 1 --2- 172.21.15.103:0/2613932240 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0eb8039300 0x7f0eb803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.743+0000 7f0ed8a00640 1 -- 172.21.15.103:0/2613932240 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0ec403e030 con 0x7f0ed4101840 2024-09-16T01:56:47.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.743+0000 7f0eca000640 1 --2- 172.21.15.103:0/2613932240 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0eb8039300 0x7f0eb803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.743+0000 7f0eca000640 1 --2- 172.21.15.103:0/2613932240 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0eb8039300 0x7f0eb803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0098c0 tx=0x7f0ebc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:47.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.747+0000 7f0ed8a00640 1 -- 172.21.15.103:0/2613932240 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ec40373b0 con 0x7f0ed4101840 2024-09-16T01:56:47.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.927+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2613932240 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ed419a5b0 con 0x7f0eb8039300 2024-09-16T01:56:47.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.928+0000 7f0ed8a00640 1 -- 172.21.15.103:0/2613932240 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f0ed419a5b0 con 0x7f0eb8039300 2024-09-16T01:56:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:47.929 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-09-16T01:56:47.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.930+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2613932240 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0eb8039300 msgr2=0x7f0eb803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.930+0000 7f0ed3e00640 1 --2- 172.21.15.103:0/2613932240 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0eb8039300 0x7f0eb803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0098c0 tx=0x7f0ebc006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.930+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2613932240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ed4101840 msgr2=0x7f0ed41991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.930+0000 7f0ed3e00640 1 --2- 172.21.15.103:0/2613932240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ed4101840 0x7f0ed41991e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009d40 tx=0x7f0ec402fc10 comp rx=0 tx=0).stop 2024-09-16T01:56:47.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.930+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2613932240 shutdown_connections 2024-09-16T01:56:47.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.930+0000 7f0ed3e00640 1 --2- 172.21.15.103:0/2613932240 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0eb8039300 0x7f0eb803b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.930+0000 7f0ed3e00640 1 --2- 172.21.15.103:0/2613932240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ed4101840 0x7f0ed41991e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.930+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2613932240 >> 172.21.15.103:0/2613932240 conn(0x7f0ed4078130 msgr2=0x7f0ed4100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:47.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.931+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2613932240 shutdown_connections 2024-09-16T01:56:47.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:47.931+0000 7f0ed3e00640 1 -- 172.21.15.103:0/2613932240 wait complete. 2024-09-16T01:56:48.552 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi172 2024-09-16T01:56:48.553 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:56:48.553 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.conf 2024-09-16T01:56:48.573 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:56:48.573 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:56:48.631 INFO:tasks.cephadm:Adding host smithi172 to orchestrator... 2024-09-16T01:56:48.632 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch host add smithi172 2024-09-16T01:56:48.993 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:49.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.445+0000 7fd60f200640 1 -- 172.21.15.103:0/1609674606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd608107660 msgr2=0x7fd608107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.445+0000 7fd60f200640 1 --2- 172.21.15.103:0/1609674606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd608107660 0x7fd608107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009880 tx=0x7fd5f802a5d0 comp rx=0 tx=0).stop 2024-09-16T01:56:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.445+0000 7fd60f200640 1 -- 172.21.15.103:0/1609674606 shutdown_connections 2024-09-16T01:56:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.445+0000 7fd60f200640 1 --2- 172.21.15.103:0/1609674606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd608107660 0x7fd608107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.445+0000 7fd60f200640 1 -- 172.21.15.103:0/1609674606 >> 172.21.15.103:0/1609674606 conn(0x7fd6080fd2b0 msgr2=0x7fd6080ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.445+0000 7fd60f200640 1 -- 172.21.15.103:0/1609674606 shutdown_connections 2024-09-16T01:56:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.445+0000 7fd60f200640 1 -- 172.21.15.103:0/1609674606 wait complete. 2024-09-16T01:56:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.446+0000 7fd60f200640 1 Processor -- start 2024-09-16T01:56:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.447+0000 7fd60f200640 1 -- start start 2024-09-16T01:56:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.447+0000 7fd60f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd608107660 0x7fd608199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.447+0000 7fd60f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd60810bc70 con 0x7fd608107660 2024-09-16T01:56:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.447+0000 7fd607e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd608107660 0x7fd608199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.447+0000 7fd607e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd608107660 0x7fd608199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42884/0 (socket says 172.21.15.103:42884) 2024-09-16T01:56:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.447+0000 7fd607e00640 1 -- 172.21.15.103:0/14078723 learned_addr learned my addr 172.21.15.103:0/14078723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.447+0000 7fd607e00640 1 -- 172.21.15.103:0/14078723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd608199880 con 0x7fd608107660 2024-09-16T01:56:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.448+0000 7fd607e00640 1 --2- 172.21.15.103:0/14078723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd608107660 0x7fd608199340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009230 tx=0x7fd5f8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.448+0000 7fd604c00640 1 -- 172.21.15.103:0/14078723 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f803f070 con 0x7fd608107660 2024-09-16T01:56:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.448+0000 7fd604c00640 1 -- 172.21.15.103:0/14078723 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f803bd40 con 0x7fd608107660 2024-09-16T01:56:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.448+0000 7fd604c00640 1 -- 172.21.15.103:0/14078723 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f8035030 con 0x7fd608107660 2024-09-16T01:56:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.448+0000 7fd60f200640 1 -- 172.21.15.103:0/14078723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd608199b70 con 0x7fd608107660 2024-09-16T01:56:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.448+0000 7fd60f200640 1 -- 172.21.15.103:0/14078723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60819dc10 con 0x7fd608107660 2024-09-16T01:56:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.449+0000 7fd604c00640 1 -- 172.21.15.103:0/14078723 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fd5f8042070 con 0x7fd608107660 2024-09-16T01:56:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.449+0000 7fd60f200640 1 -- 172.21.15.103:0/14078723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd608102d10 con 0x7fd608107660 2024-09-16T01:56:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.450+0000 7fd604c00640 1 --2- 172.21.15.103:0/14078723 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd5ec039300 0x7fd5ec03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.450+0000 7fd604c00640 1 -- 172.21.15.103:0/14078723 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd5f806cf60 con 0x7fd608107660 2024-09-16T01:56:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.450+0000 7fd607400640 1 --2- 172.21.15.103:0/14078723 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd5ec039300 0x7fd5ec03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.450+0000 7fd607400640 1 --2- 172.21.15.103:0/14078723 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd5ec039300 0x7fd5ec03b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40098c0 tx=0x7fd5f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:49.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.455+0000 7fd604c00640 1 -- 172.21.15.103:0/14078723 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5f8030d60 con 0x7fd608107660 2024-09-16T01:56:49.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:49.636+0000 7fd60f200640 1 -- 172.21.15.103:0/14078723 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}) -- 0x7fd60819a6a0 con 0x7fd5ec039300 2024-09-16T01:56:49.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:49 smithi103 ceph-mon[30398]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:56:49.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:49 smithi103 ceph-mon[30398]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:56:51.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: Deploying cephadm binary to smithi172 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: Updating smithi136:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:56:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:51.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:51.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:52.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:52 smithi103 ceph-mon[30398]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:56:52.758 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:52 smithi103 ceph-mon[30398]: Updating smithi136:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:56:52.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.811+0000 7fd604c00640 1 -- 172.21.15.103:0/14078723 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd60819a6a0 con 0x7fd5ec039300 2024-09-16T01:56:52.813 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi172' with addr '172.21.15.172' 2024-09-16T01:56:52.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.814+0000 7fd60f200640 1 -- 172.21.15.103:0/14078723 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd5ec039300 msgr2=0x7fd5ec03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:52.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.814+0000 7fd60f200640 1 --2- 172.21.15.103:0/14078723 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd5ec039300 0x7fd5ec03b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40098c0 tx=0x7fd5f4006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:52.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.814+0000 7fd60f200640 1 -- 172.21.15.103:0/14078723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd608107660 msgr2=0x7fd608199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:52.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.814+0000 7fd60f200640 1 --2- 172.21.15.103:0/14078723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd608107660 0x7fd608199340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009230 tx=0x7fd5f8003d80 comp rx=0 tx=0).stop 2024-09-16T01:56:52.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.814+0000 7fd60f200640 1 -- 172.21.15.103:0/14078723 shutdown_connections 2024-09-16T01:56:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.814+0000 7fd60f200640 1 --2- 172.21.15.103:0/14078723 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd5ec039300 0x7fd5ec03b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.814+0000 7fd60f200640 1 --2- 172.21.15.103:0/14078723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd608107660 0x7fd608199340 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.814+0000 7fd60f200640 1 -- 172.21.15.103:0/14078723 >> 172.21.15.103:0/14078723 conn(0x7fd6080fd2b0 msgr2=0x7fd608075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.814+0000 7fd60f200640 1 -- 172.21.15.103:0/14078723 shutdown_connections 2024-09-16T01:56:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:52.814+0000 7fd60f200640 1 -- 172.21.15.103:0/14078723 wait complete. 2024-09-16T01:56:53.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:53 smithi103 ceph-mon[30398]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:56:53.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:53.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:53.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:53.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T01:56:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:54.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.377+0000 7f490ea00640 1 -- 172.21.15.103:0/2548875804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49081015e0 msgr2=0x7f49081019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:54.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.377+0000 7f490ea00640 1 --2- 172.21.15.103:0/2548875804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49081015e0 0x7f49081019c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009880 tx=0x7f48fc02a5d0 comp rx=0 tx=0).stop 2024-09-16T01:56:54.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.377+0000 7f490ea00640 1 -- 172.21.15.103:0/2548875804 shutdown_connections 2024-09-16T01:56:54.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.377+0000 7f490ea00640 1 --2- 172.21.15.103:0/2548875804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49081015e0 0x7f49081019c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:54.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.377+0000 7f490ea00640 1 -- 172.21.15.103:0/2548875804 >> 172.21.15.103:0/2548875804 conn(0x7f4908078240 msgr2=0x7f49080ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:54.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.378+0000 7f490ea00640 1 -- 172.21.15.103:0/2548875804 shutdown_connections 2024-09-16T01:56:54.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.378+0000 7f490ea00640 1 -- 172.21.15.103:0/2548875804 wait complete. 2024-09-16T01:56:54.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.378+0000 7f490ea00640 1 Processor -- start 2024-09-16T01:56:54.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.379+0000 7f490ea00640 1 -- start start 2024-09-16T01:56:54.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.379+0000 7f490ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49081015e0 0x7f49080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:54.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.379+0000 7f490ea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f490810ba30 con 0x7f49081015e0 2024-09-16T01:56:54.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.379+0000 7f490d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49081015e0 0x7f49080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:54.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.379+0000 7f490d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49081015e0 0x7f49080744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42894/0 (socket says 172.21.15.103:42894) 2024-09-16T01:56:54.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.379+0000 7f490d600640 1 -- 172.21.15.103:0/1776628485 learned_addr learned my addr 172.21.15.103:0/1776628485 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:54.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.380+0000 7f490d600640 1 -- 172.21.15.103:0/1776628485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49080763e0 con 0x7f49081015e0 2024-09-16T01:56:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.380+0000 7f490d600640 1 --2- 172.21.15.103:0/1776628485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49081015e0 0x7f49080744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009230 tx=0x7f48fc003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.380+0000 7f4906000640 1 -- 172.21.15.103:0/1776628485 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48fc03f070 con 0x7f49081015e0 2024-09-16T01:56:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.380+0000 7f4906000640 1 -- 172.21.15.103:0/1776628485 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc03bd40 con 0x7f49081015e0 2024-09-16T01:56:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.380+0000 7f4906000640 1 -- 172.21.15.103:0/1776628485 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48fc035030 con 0x7f49081015e0 2024-09-16T01:56:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.380+0000 7f490ea00640 1 -- 172.21.15.103:0/1776628485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4908074a90 con 0x7f49081015e0 2024-09-16T01:56:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.380+0000 7f490ea00640 1 -- 172.21.15.103:0/1776628485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4908074f50 con 0x7f49081015e0 2024-09-16T01:56:54.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.381+0000 7f4906000640 1 -- 172.21.15.103:0/1776628485 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f48fc042070 con 0x7f49081015e0 2024-09-16T01:56:54.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.381+0000 7f490ea00640 1 -- 172.21.15.103:0/1776628485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4908102ac0 con 0x7f49081015e0 2024-09-16T01:56:54.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.382+0000 7f4906000640 1 --2- 172.21.15.103:0/1776628485 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48f0039300 0x7f48f003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:54.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.382+0000 7f4906000640 1 -- 172.21.15.103:0/1776628485 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48fc06cbe0 con 0x7f49081015e0 2024-09-16T01:56:54.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.382+0000 7f490cc00640 1 --2- 172.21.15.103:0/1776628485 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48f0039300 0x7f48f003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:54.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.383+0000 7f490cc00640 1 --2- 172.21.15.103:0/1776628485 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48f0039300 0x7f48f003b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f48f40098c0 tx=0x7f48f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:54.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.387+0000 7f4906000640 1 -- 172.21.15.103:0/1776628485 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48fc044e10 con 0x7f49081015e0 2024-09-16T01:56:54.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.567+0000 7f490ea00640 1 -- 172.21.15.103:0/1776628485 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4908075a90 con 0x7f48f0039300 2024-09-16T01:56:54.569 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:54 smithi103 ceph-mon[30398]: Added host smithi172 2024-09-16T01:56:54.569 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:54 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:54.569 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:54 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:54.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.568+0000 7f4906000640 1 -- 172.21.15.103:0/1776628485 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f4908075a90 con 0x7f48f0039300 2024-09-16T01:56:54.569 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:56:54.570 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}, {"addr": "172.21.15.172", "hostname": "smithi172", "labels": [], "status": ""}] 2024-09-16T01:56:54.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.571+0000 7f490ea00640 1 -- 172.21.15.103:0/1776628485 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48f0039300 msgr2=0x7f48f003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:54.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.571+0000 7f490ea00640 1 --2- 172.21.15.103:0/1776628485 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48f0039300 0x7f48f003b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f48f40098c0 tx=0x7f48f4006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:54.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.571+0000 7f490ea00640 1 -- 172.21.15.103:0/1776628485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49081015e0 msgr2=0x7f49080744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:54.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.571+0000 7f490ea00640 1 --2- 172.21.15.103:0/1776628485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49081015e0 0x7f49080744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009230 tx=0x7f48fc003d80 comp rx=0 tx=0).stop 2024-09-16T01:56:54.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.571+0000 7f490ea00640 1 -- 172.21.15.103:0/1776628485 shutdown_connections 2024-09-16T01:56:54.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.571+0000 7f490ea00640 1 --2- 172.21.15.103:0/1776628485 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f48f0039300 0x7f48f003b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:54.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.571+0000 7f490ea00640 1 --2- 172.21.15.103:0/1776628485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49081015e0 0x7f49080744f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:54.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.571+0000 7f490ea00640 1 -- 172.21.15.103:0/1776628485 >> 172.21.15.103:0/1776628485 conn(0x7f4908078240 msgr2=0x7f4908108f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:54.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.571+0000 7f490ea00640 1 -- 172.21.15.103:0/1776628485 shutdown_connections 2024-09-16T01:56:54.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:54.571+0000 7f490ea00640 1 -- 172.21.15.103:0/1776628485 wait complete. 2024-09-16T01:56:55.167 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T01:56:55.167 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T01:56:55.526 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:55.664 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:55 smithi103 ceph-mon[30398]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:56:55.664 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:55 smithi103 ceph-mon[30398]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:56:56.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.075+0000 7f573a400640 1 -- 172.21.15.103:0/3389569230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5734101840 msgr2=0x7f5734101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:56.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.075+0000 7f573a400640 1 --2- 172.21.15.103:0/3389569230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5734101840 0x7f5734101c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5724009880 tx=0x7f572402a5d0 comp rx=0 tx=0).stop 2024-09-16T01:56:56.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.075+0000 7f573a400640 1 -- 172.21.15.103:0/3389569230 shutdown_connections 2024-09-16T01:56:56.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.075+0000 7f573a400640 1 --2- 172.21.15.103:0/3389569230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5734101840 0x7f5734101c20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:56.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.075+0000 7f573a400640 1 -- 172.21.15.103:0/3389569230 >> 172.21.15.103:0/3389569230 conn(0x7f5734078130 msgr2=0x7f5734078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:56.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.075+0000 7f573a400640 1 -- 172.21.15.103:0/3389569230 shutdown_connections 2024-09-16T01:56:56.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.075+0000 7f573a400640 1 -- 172.21.15.103:0/3389569230 wait complete. 2024-09-16T01:56:56.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.076+0000 7f573a400640 1 Processor -- start 2024-09-16T01:56:56.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.076+0000 7f573a400640 1 -- start start 2024-09-16T01:56:56.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.076+0000 7f573a400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5734101840 0x7f57340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.076+0000 7f573a400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f573410ba50 con 0x7f5734101840 2024-09-16T01:56:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.076+0000 7f5739000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5734101840 0x7f57340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.076+0000 7f5739000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5734101840 0x7f57340744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42908/0 (socket says 172.21.15.103:42908) 2024-09-16T01:56:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.077+0000 7f5739000640 1 -- 172.21.15.103:0/1819527321 learned_addr learned my addr 172.21.15.103:0/1819527321 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:56:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.077+0000 7f5739000640 1 -- 172.21.15.103:0/1819527321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5734076390 con 0x7f5734101840 2024-09-16T01:56:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.077+0000 7f5739000640 1 --2- 172.21.15.103:0/1819527321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5734101840 0x7f57340744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5724009230 tx=0x7f5724003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.077+0000 7f5732000640 1 -- 172.21.15.103:0/1819527321 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f572403f070 con 0x7f5734101840 2024-09-16T01:56:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.077+0000 7f5732000640 1 -- 172.21.15.103:0/1819527321 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572403bd40 con 0x7f5734101840 2024-09-16T01:56:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.077+0000 7f573a400640 1 -- 172.21.15.103:0/1819527321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5734074a30 con 0x7f5734101840 2024-09-16T01:56:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.077+0000 7f5732000640 1 -- 172.21.15.103:0/1819527321 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5724035030 con 0x7f5734101840 2024-09-16T01:56:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.078+0000 7f573a400640 1 -- 172.21.15.103:0/1819527321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5734074ef0 con 0x7f5734101840 2024-09-16T01:56:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.079+0000 7f5732000640 1 -- 172.21.15.103:0/1819527321 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f5724042070 con 0x7f5734101840 2024-09-16T01:56:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.079+0000 7f573a400640 1 -- 172.21.15.103:0/1819527321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5734102af0 con 0x7f5734101840 2024-09-16T01:56:56.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.079+0000 7f5732000640 1 --2- 172.21.15.103:0/1819527321 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f57180392a0 0x7f571803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:56.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.079+0000 7f5732000640 1 -- 172.21.15.103:0/1819527321 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f572406cf70 con 0x7f5734101840 2024-09-16T01:56:56.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.079+0000 7f572be00640 1 --2- 172.21.15.103:0/1819527321 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f57180392a0 0x7f571803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:56.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.080+0000 7f572be00640 1 --2- 172.21.15.103:0/1819527321 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f57180392a0 0x7f571803b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f571c0098c0 tx=0x7f571c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:56.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.084+0000 7f5732000640 1 -- 172.21.15.103:0/1819527321 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f572403b860 con 0x7f5734101840 2024-09-16T01:56:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.266+0000 7f573a400640 1 -- 172.21.15.103:0/1819527321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5734075a20 con 0x7f5734101840 2024-09-16T01:56:56.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.394+0000 7f5732000640 1 -- 172.21.15.103:0/1819527321 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f57240314c0 con 0x7f5734101840 2024-09-16T01:56:56.395 INFO:teuthology.orchestra.run.smithi103.stderr:adjusted tunables profile to default 2024-09-16T01:56:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.396+0000 7f573a400640 1 -- 172.21.15.103:0/1819527321 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f57180392a0 msgr2=0x7f571803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.396+0000 7f573a400640 1 --2- 172.21.15.103:0/1819527321 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f57180392a0 0x7f571803b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f571c0098c0 tx=0x7f571c006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.397+0000 7f573a400640 1 -- 172.21.15.103:0/1819527321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5734101840 msgr2=0x7f57340744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.397+0000 7f573a400640 1 --2- 172.21.15.103:0/1819527321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5734101840 0x7f57340744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5724009230 tx=0x7f5724003d80 comp rx=0 tx=0).stop 2024-09-16T01:56:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.397+0000 7f573a400640 1 -- 172.21.15.103:0/1819527321 shutdown_connections 2024-09-16T01:56:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.397+0000 7f573a400640 1 --2- 172.21.15.103:0/1819527321 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f57180392a0 0x7f571803b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.397+0000 7f573a400640 1 --2- 172.21.15.103:0/1819527321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5734101840 0x7f57340744f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.397+0000 7f573a400640 1 -- 172.21.15.103:0/1819527321 >> 172.21.15.103:0/1819527321 conn(0x7f5734078130 msgr2=0x7f5734102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.397+0000 7f573a400640 1 -- 172.21.15.103:0/1819527321 shutdown_connections 2024-09-16T01:56:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:56:56.397+0000 7f573a400640 1 -- 172.21.15.103:0/1819527321 wait complete. 2024-09-16T01:56:56.633 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:56 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1819527321' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T01:56:57.090 INFO:tasks.cephadm:Adding mon.a on smithi103 2024-09-16T01:56:57.090 INFO:tasks.cephadm:Adding mon.b on smithi136 2024-09-16T01:56:57.090 INFO:tasks.cephadm:Adding mon.c on smithi172 2024-09-16T01:56:57.090 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi103:172.21.15.103=a;smithi136:172.21.15.136=b;smithi172:172.21.15.172=c' 2024-09-16T01:56:57.335 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:56:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:57 smithi103 ceph-mon[30398]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:56:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:57 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1819527321' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T01:56:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:57 smithi103 ceph-mon[30398]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:56:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:56:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:57.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.863+0000 7ffb3e49e640 1 -- 172.21.15.172:0/1438703157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38107690 msgr2=0x7ffb38107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:57.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.863+0000 7ffb3e49e640 1 --2- 172.21.15.172:0/1438703157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38107690 0x7ffb38107a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009880 tx=0x7ffb2802a5d0 comp rx=0 tx=0).stop 2024-09-16T01:56:57.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.863+0000 7ffb3e49e640 1 -- 172.21.15.172:0/1438703157 shutdown_connections 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.863+0000 7ffb3e49e640 1 --2- 172.21.15.172:0/1438703157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38107690 0x7ffb38107a70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.863+0000 7ffb3e49e640 1 -- 172.21.15.172:0/1438703157 >> 172.21.15.172:0/1438703157 conn(0x7ffb38076250 msgr2=0x7ffb38076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.863+0000 7ffb3e49e640 1 -- 172.21.15.172:0/1438703157 shutdown_connections 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.863+0000 7ffb3e49e640 1 -- 172.21.15.172:0/1438703157 wait complete. 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.864+0000 7ffb3e49e640 1 Processor -- start 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.864+0000 7ffb3e49e640 1 -- start start 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.864+0000 7ffb3e49e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38107690 0x7ffb38100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.864+0000 7ffb3e49e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3810bce0 con 0x7ffb38107690 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.865+0000 7ffb37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38107690 0x7ffb38100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.865+0000 7ffb37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38107690 0x7ffb38100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.172:41804/0 (socket says 172.21.15.172:41804) 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.865+0000 7ffb37fff640 1 -- 172.21.15.172:0/179065765 learned_addr learned my addr 172.21.15.172:0/179065765 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T01:56:57.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.865+0000 7ffb37fff640 1 -- 172.21.15.172:0/179065765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb381026f0 con 0x7ffb38107690 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.865+0000 7ffb37fff640 1 --2- 172.21.15.172:0/179065765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38107690 0x7ffb38100800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009230 tx=0x7ffb28003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.865+0000 7ffb35ffb640 1 -- 172.21.15.172:0/179065765 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb2803f070 con 0x7ffb38107690 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.865+0000 7ffb3e49e640 1 -- 172.21.15.172:0/179065765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb38100d40 con 0x7ffb38107690 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.866+0000 7ffb3e49e640 1 -- 172.21.15.172:0/179065765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb38101200 con 0x7ffb38107690 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.866+0000 7ffb35ffb640 1 -- 172.21.15.172:0/179065765 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb2803bc80 con 0x7ffb38107690 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.866+0000 7ffb35ffb640 1 -- 172.21.15.172:0/179065765 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb28049060 con 0x7ffb38107690 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.866+0000 7ffb35ffb640 1 -- 172.21.15.172:0/179065765 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ffb28042070 con 0x7ffb38107690 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.866+0000 7ffb3e49e640 1 -- 172.21.15.172:0/179065765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb18005290 con 0x7ffb38107690 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.867+0000 7ffb35ffb640 1 --2- 172.21.15.172:0/179065765 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb240392a0 0x7ffb2403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.867+0000 7ffb35ffb640 1 -- 172.21.15.172:0/179065765 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffb2806d000 con 0x7ffb38107690 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.867+0000 7ffb2ffff640 1 --2- 172.21.15.172:0/179065765 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb240392a0 0x7ffb2403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:57.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.868+0000 7ffb2ffff640 1 --2- 172.21.15.172:0/179065765 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb240392a0 0x7ffb2403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffb200098c0 tx=0x7ffb20006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:57.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:57.872+0000 7ffb35ffb640 1 -- 172.21.15.172:0/179065765 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb28042360 con 0x7ffb38107690 2024-09-16T01:56:58.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.052+0000 7ffb3e49e640 1 -- 172.21.15.172:0/179065765 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi136:172.21.15.136=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}) -- 0x7ffb18000c80 con 0x7ffb240392a0 2024-09-16T01:56:58.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.059+0000 7ffb35ffb640 1 -- 172.21.15.172:0/179065765 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ffb18000c80 con 0x7ffb240392a0 2024-09-16T01:56:58.060 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mon update... 2024-09-16T01:56:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.061+0000 7ffb3e49e640 1 -- 172.21.15.172:0/179065765 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb240392a0 msgr2=0x7ffb2403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.061+0000 7ffb3e49e640 1 --2- 172.21.15.172:0/179065765 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb240392a0 0x7ffb2403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffb200098c0 tx=0x7ffb20006db0 comp rx=0 tx=0).stop 2024-09-16T01:56:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.061+0000 7ffb3e49e640 1 -- 172.21.15.172:0/179065765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38107690 msgr2=0x7ffb38100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.061+0000 7ffb3e49e640 1 --2- 172.21.15.172:0/179065765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38107690 0x7ffb38100800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009230 tx=0x7ffb28003d40 comp rx=0 tx=0).stop 2024-09-16T01:56:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.061+0000 7ffb3e49e640 1 -- 172.21.15.172:0/179065765 shutdown_connections 2024-09-16T01:56:58.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.061+0000 7ffb3e49e640 1 --2- 172.21.15.172:0/179065765 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb240392a0 0x7ffb2403b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:58.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.061+0000 7ffb3e49e640 1 --2- 172.21.15.172:0/179065765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38107690 0x7ffb38100800 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:58.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.061+0000 7ffb3e49e640 1 -- 172.21.15.172:0/179065765 >> 172.21.15.172:0/179065765 conn(0x7ffb38076250 msgr2=0x7ffb380fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:58.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.061+0000 7ffb3e49e640 1 -- 172.21.15.172:0/179065765 shutdown_connections 2024-09-16T01:56:58.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:56:58.061+0000 7ffb3e49e640 1 -- 172.21.15.172:0/179065765 wait complete. 2024-09-16T01:56:58.664 DEBUG:teuthology.orchestra.run.smithi136:mon.b> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T01:56:58.667 DEBUG:teuthology.orchestra.run.smithi172:mon.c> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T01:56:58.670 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:56:58.670 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:56:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-16T01:56:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: Updating smithi172:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:56:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:56:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: Updating smithi172:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:56:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:56:58.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:58.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:58.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:56:58.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:56:58.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:59 smithi103 ceph-mon[30398]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi136:172.21.15.136=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:56:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:59 smithi103 ceph-mon[30398]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi136:172.21.15.136=b;smithi172:172.21.15.172=c;count:3 2024-09-16T01:56:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:59 smithi103 ceph-mon[30398]: Deploying daemon mon.c on smithi172 2024-09-16T01:56:59.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:56:59 smithi103 ceph-mon[30398]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:56:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:01.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.433+0000 7f8074cf1640 1 -- 172.21.15.172:0/58959566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070106590 msgr2=0x7f8070106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:01.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.433+0000 7f8074cf1640 1 --2- 172.21.15.172:0/58959566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070106590 0x7f8070106970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8060009880 tx=0x7f806002a5d0 comp rx=0 tx=0).stop 2024-09-16T01:57:01.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.433+0000 7f8074cf1640 1 -- 172.21.15.172:0/58959566 shutdown_connections 2024-09-16T01:57:01.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.433+0000 7f8074cf1640 1 --2- 172.21.15.172:0/58959566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070106590 0x7f8070106970 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:01.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.433+0000 7f8074cf1640 1 -- 172.21.15.172:0/58959566 >> 172.21.15.172:0/58959566 conn(0x7f80700fc1a0 msgr2=0x7f80700fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:01.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.434+0000 7f8074cf1640 1 -- 172.21.15.172:0/58959566 shutdown_connections 2024-09-16T01:57:01.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.434+0000 7f8074cf1640 1 -- 172.21.15.172:0/58959566 wait complete. 2024-09-16T01:57:01.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.435+0000 7f8074cf1640 1 Processor -- start 2024-09-16T01:57:01.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.435+0000 7f8074cf1640 1 -- start start 2024-09-16T01:57:01.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.435+0000 7f8074cf1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070106590 0x7f8070195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:01.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.435+0000 7f8074cf1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f807010abc0 con 0x7f8070106590 2024-09-16T01:57:01.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.436+0000 7f806e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070106590 0x7f8070195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:01.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.436+0000 7f806e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070106590 0x7f8070195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.172:41836/0 (socket says 172.21.15.172:41836) 2024-09-16T01:57:01.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.436+0000 7f806e59c640 1 -- 172.21.15.172:0/2856748963 learned_addr learned my addr 172.21.15.172:0/2856748963 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T01:57:01.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.436+0000 7f806e59c640 1 -- 172.21.15.172:0/2856748963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8070196530 con 0x7f8070106590 2024-09-16T01:57:01.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.436+0000 7f806e59c640 1 --2- 172.21.15.172:0/2856748963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070106590 0x7f8070195ff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8060009230 tx=0x7f8060003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:01.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.436+0000 7f8067fff640 1 -- 172.21.15.172:0/2856748963 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f806003f070 con 0x7f8070106590 2024-09-16T01:57:01.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.437+0000 7f8074cf1640 1 -- 172.21.15.172:0/2856748963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80701967c0 con 0x7f8070106590 2024-09-16T01:57:01.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.437+0000 7f8067fff640 1 -- 172.21.15.172:0/2856748963 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f806003bc80 con 0x7f8070106590 2024-09-16T01:57:01.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.437+0000 7f8067fff640 1 -- 172.21.15.172:0/2856748963 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8060035030 con 0x7f8070106590 2024-09-16T01:57:01.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.437+0000 7f8074cf1640 1 -- 172.21.15.172:0/2856748963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807019a8c0 con 0x7f8070106590 2024-09-16T01:57:01.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.437+0000 7f8067fff640 1 -- 172.21.15.172:0/2856748963 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f8060042070 con 0x7f8070106590 2024-09-16T01:57:01.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.438+0000 7f8074cf1640 1 -- 172.21.15.172:0/2856748963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8070101c00 con 0x7f8070106590 2024-09-16T01:57:01.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.438+0000 7f8067fff640 1 --2- 172.21.15.172:0/2856748963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f805c0392a0 0x7f805c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:01.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.438+0000 7f8067fff640 1 -- 172.21.15.172:0/2856748963 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f806006d060 con 0x7f8070106590 2024-09-16T01:57:01.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.438+0000 7f8065d9b640 1 --2- 172.21.15.172:0/2856748963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f805c0392a0 0x7f805c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:01.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.439+0000 7f8065d9b640 1 --2- 172.21.15.172:0/2856748963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f805c0392a0 0x7f805c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80580098c0 tx=0x7f8058006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:01.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.443+0000 7f8067fff640 1 -- 172.21.15.172:0/2856748963 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f806004c350 con 0x7f8070106590 2024-09-16T01:57:01.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.699+0000 7f8074cf1640 1 -- 172.21.15.172:0/2856748963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8070197350 con 0x7f8070106590 2024-09-16T01:57:01.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.700+0000 7f8067fff640 1 -- 172.21.15.172:0/2856748963 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f8060031e20 con 0x7f8070106590 2024-09-16T01:57:01.701 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:57:01.701 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":1,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","modified":"2024-09-16T01:55:32.719520Z","created":"2024-09-16T01:55:32.719520Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T01:57:01.701 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 1 2024-09-16T01:57:01.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.702+0000 7f8074cf1640 1 -- 172.21.15.172:0/2856748963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f805c0392a0 msgr2=0x7f805c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:01.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.702+0000 7f8074cf1640 1 --2- 172.21.15.172:0/2856748963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f805c0392a0 0x7f805c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80580098c0 tx=0x7f8058006db0 comp rx=0 tx=0).stop 2024-09-16T01:57:01.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.702+0000 7f8074cf1640 1 -- 172.21.15.172:0/2856748963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070106590 msgr2=0x7f8070195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:01.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.702+0000 7f8074cf1640 1 --2- 172.21.15.172:0/2856748963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070106590 0x7f8070195ff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8060009230 tx=0x7f8060003d40 comp rx=0 tx=0).stop 2024-09-16T01:57:01.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.702+0000 7f8074cf1640 1 -- 172.21.15.172:0/2856748963 shutdown_connections 2024-09-16T01:57:01.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.702+0000 7f8074cf1640 1 --2- 172.21.15.172:0/2856748963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f805c0392a0 0x7f805c03b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:01.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.702+0000 7f8074cf1640 1 --2- 172.21.15.172:0/2856748963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070106590 0x7f8070195ff0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:01.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.702+0000 7f8074cf1640 1 -- 172.21.15.172:0/2856748963 >> 172.21.15.172:0/2856748963 conn(0x7f80700fc1a0 msgr2=0x7f80700fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:01.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.702+0000 7f8074cf1640 1 -- 172.21.15.172:0/2856748963 shutdown_connections 2024-09-16T01:57:01.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:01.702+0000 7f8074cf1640 1 -- 172.21.15.172:0/2856748963 wait complete. 2024-09-16T01:57:01.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:01 smithi103 ceph-mon[30398]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:02.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 podman[35321]: 2024-09-16 01:57:02.49807653 +0000 UTC m=+0.125931765 container create 9e5b8f07d99b33ea7b16072cc45426c2d39c19a7bd488b370303ada7e7a2bb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:57:02.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:02 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/2856748963' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:03.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 podman[35321]: 2024-09-16 01:57:02.691899358 +0000 UTC m=+0.319754596 container init 9e5b8f07d99b33ea7b16072cc45426c2d39c19a7bd488b370303ada7e7a2bb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T01:57:03.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 podman[35321]: 2024-09-16 01:57:02.695252161 +0000 UTC m=+0.323107406 container start 9e5b8f07d99b33ea7b16072cc45426c2d39c19a7bd488b370303ada7e7a2bb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T01:57:03.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T01:57:03.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T01:57:03.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: pidfile_write: ignore empty --pid-file 2024-09-16T01:57:03.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: load: jerasure load: lrc 2024-09-16T01:57:03.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: RocksDB version: 7.9.2 2024-09-16T01:57:03.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Git sha 0 2024-09-16T01:57:03.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: DB SUMMARY 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: DB Session ID: T8Q67UPOOI1GDLDVPQ02 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: CURRENT file: CURRENT 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.error_if_exists: 0 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.create_if_missing: 0 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.paranoid_checks: 1 2024-09-16T01:57:03.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T01:57:03.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T01:57:03.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T01:57:03.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.env: 0x55d140a59940 2024-09-16T01:57:03.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T01:57:03.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.info_log: 0x55d14295d6e0 2024-09-16T01:57:03.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T01:57:03.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.statistics: (nil) 2024-09-16T01:57:03.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.use_fsync: 0 2024-09-16T01:57:03.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_log_file_size: 0 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.allow_fallocate: 1 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.use_direct_reads: 0 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.db_log_dir: 2024-09-16T01:57:03.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.wal_dir: 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.write_buffer_manager: 0x55d14296cdc0 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T01:57:03.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T01:57:03.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T01:57:03.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T01:57:03.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T01:57:03.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.unordered_write: 0 2024-09-16T01:57:03.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T01:57:03.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T01:57:03.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T01:57:03.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T01:57:03.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.row_cache: None 2024-09-16T01:57:03.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.wal_filter: None 2024-09-16T01:57:03.053 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T01:57:03.053 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T01:57:03.053 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.two_write_queues: 0 2024-09-16T01:57:03.054 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T01:57:03.054 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.wal_compression: 0 2024-09-16T01:57:03.054 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.atomic_flush: 0 2024-09-16T01:57:03.054 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T01:57:03.054 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T01:57:03.055 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T01:57:03.055 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.log_readahead_size: 0 2024-09-16T01:57:03.055 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T01:57:03.055 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T01:57:03.055 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T01:57:03.055 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T01:57:03.055 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T01:57:03.055 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T01:57:03.056 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T01:57:03.056 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_background_jobs: 2 2024-09-16T01:57:03.056 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_background_compactions: -1 2024-09-16T01:57:03.056 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_subcompactions: 1 2024-09-16T01:57:03.056 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T01:57:03.056 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T01:57:03.056 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T01:57:03.056 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T01:57:03.057 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T01:57:03.057 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T01:57:03.057 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T01:57:03.057 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T01:57:03.057 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_open_files: -1 2024-09-16T01:57:03.057 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T01:57:03.057 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T01:57:03.057 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T01:57:03.058 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T01:57:03.058 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_background_flushes: -1 2024-09-16T01:57:03.058 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Compression algorithms supported: 2024-09-16T01:57:03.058 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: kZSTD supported: 0 2024-09-16T01:57:03.058 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: kXpressCompression supported: 0 2024-09-16T01:57:03.059 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: kBZip2Compression supported: 0 2024-09-16T01:57:03.059 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T01:57:03.059 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: kLZ4Compression supported: 1 2024-09-16T01:57:03.059 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: kZlibCompression supported: 1 2024-09-16T01:57:03.059 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T01:57:03.059 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: kSnappyCompression supported: 1 2024-09-16T01:57:03.059 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T01:57:03.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T01:57:03.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-16T01:57:03.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T01:57:03.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T01:57:03.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.merge_operator: 2024-09-16T01:57:03.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_filter: None 2024-09-16T01:57:03.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T01:57:03.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T01:57:03.061 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T01:57:03.061 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T01:57:03.061 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d14295d320) 2024-09-16T01:57:03.061 INFO:journalctl@ceph.mon.c.smithi172.stdout: cache_index_and_filter_blocks: 1 2024-09-16T01:57:03.062 INFO:journalctl@ceph.mon.c.smithi172.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T01:57:03.062 INFO:journalctl@ceph.mon.c.smithi172.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T01:57:03.062 INFO:journalctl@ceph.mon.c.smithi172.stdout: pin_top_level_index_and_filter: 1 2024-09-16T01:57:03.062 INFO:journalctl@ceph.mon.c.smithi172.stdout: index_type: 0 2024-09-16T01:57:03.062 INFO:journalctl@ceph.mon.c.smithi172.stdout: data_block_index_type: 0 2024-09-16T01:57:03.062 INFO:journalctl@ceph.mon.c.smithi172.stdout: index_shortening: 1 2024-09-16T01:57:03.063 INFO:journalctl@ceph.mon.c.smithi172.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T01:57:03.063 INFO:journalctl@ceph.mon.c.smithi172.stdout: checksum: 4 2024-09-16T01:57:03.063 INFO:journalctl@ceph.mon.c.smithi172.stdout: no_block_cache: 0 2024-09-16T01:57:03.063 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache: 0x55d1429511f0 2024-09-16T01:57:03.063 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache_name: BinnedLRUCache 2024-09-16T01:57:03.063 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache_options: 2024-09-16T01:57:03.064 INFO:journalctl@ceph.mon.c.smithi172.stdout: capacity : 536870912 2024-09-16T01:57:03.064 INFO:journalctl@ceph.mon.c.smithi172.stdout: num_shard_bits : 4 2024-09-16T01:57:03.064 INFO:journalctl@ceph.mon.c.smithi172.stdout: strict_capacity_limit : 0 2024-09-16T01:57:03.064 INFO:journalctl@ceph.mon.c.smithi172.stdout: high_pri_pool_ratio: 0.000 2024-09-16T01:57:03.064 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache_compressed: (nil) 2024-09-16T01:57:03.065 INFO:journalctl@ceph.mon.c.smithi172.stdout: persistent_cache: (nil) 2024-09-16T01:57:03.065 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_size: 4096 2024-09-16T01:57:03.065 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_size_deviation: 10 2024-09-16T01:57:03.065 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_restart_interval: 16 2024-09-16T01:57:03.065 INFO:journalctl@ceph.mon.c.smithi172.stdout: index_block_restart_interval: 1 2024-09-16T01:57:03.065 INFO:journalctl@ceph.mon.c.smithi172.stdout: metadata_block_size: 4096 2024-09-16T01:57:03.066 INFO:journalctl@ceph.mon.c.smithi172.stdout: partition_filters: 0 2024-09-16T01:57:03.066 INFO:journalctl@ceph.mon.c.smithi172.stdout: use_delta_encoding: 1 2024-09-16T01:57:03.066 INFO:journalctl@ceph.mon.c.smithi172.stdout: filter_policy: bloomfilter 2024-09-16T01:57:03.066 INFO:journalctl@ceph.mon.c.smithi172.stdout: whole_key_filtering: 1 2024-09-16T01:57:03.066 INFO:journalctl@ceph.mon.c.smithi172.stdout: verify_compression: 0 2024-09-16T01:57:03.066 INFO:journalctl@ceph.mon.c.smithi172.stdout: read_amp_bytes_per_bit: 0 2024-09-16T01:57:03.066 INFO:journalctl@ceph.mon.c.smithi172.stdout: format_version: 5 2024-09-16T01:57:03.066 INFO:journalctl@ceph.mon.c.smithi172.stdout: enable_index_compression: 1 2024-09-16T01:57:03.066 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_align: 0 2024-09-16T01:57:03.067 INFO:journalctl@ceph.mon.c.smithi172.stdout: max_auto_readahead_size: 262144 2024-09-16T01:57:03.067 INFO:journalctl@ceph.mon.c.smithi172.stdout: prepopulate_block_cache: 0 2024-09-16T01:57:03.068 INFO:journalctl@ceph.mon.c.smithi172.stdout: initial_auto_readahead_size: 8192 2024-09-16T01:57:03.068 INFO:journalctl@ceph.mon.c.smithi172.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T01:57:03.068 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T01:57:03.069 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T01:57:03.069 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compression: NoCompression 2024-09-16T01:57:03.069 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T01:57:03.069 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T01:57:03.069 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T01:57:03.069 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.num_levels: 7 2024-09-16T01:57:03.070 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T01:57:03.070 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T01:57:03.070 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T01:57:03.070 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T01:57:03.070 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T01:57:03.070 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T01:57:03.070 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T01:57:03.071 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:57:03.071 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T01:57:03.071 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T01:57:03.071 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:57:03.071 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:57:03.071 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T01:57:03.072 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T01:57:03.072 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T01:57:03.072 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T01:57:03.072 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:57:03.072 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:57:03.072 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T01:57:03.073 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T01:57:03.073 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:57:03.073 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T01:57:03.073 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T01:57:03.073 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T01:57:03.073 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T01:57:03.073 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T01:57:03.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T01:57:03.075 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.inplace_update_support: 0 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.bloom_locality: 0 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.max_successive_merges: 0 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T01:57:03.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.ttl: 2592000 2024-09-16T01:57:03.077 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T01:57:03.077 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T01:57:03.077 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T01:57:03.077 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.enable_blob_files: false 2024-09-16T01:57:03.077 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.min_blob_size: 0 2024-09-16T01:57:03.077 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8e2d5f52-9504-482a-813f-45a1a39a850e 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451822722155, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T01:57:03.078 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451822722556, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726451822, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8e2d5f52-9504-482a-813f-45a1a39a850e", "db_session_id": "T8Q67UPOOI1GDLDVPQ02", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451822722648, "job": 1, "event": "recovery_finished"} 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d14297ee00 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: DB pointer 0x55d142a7a000 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** DB Stats ** 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:57:03.079 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** Compaction Stats [default] ** 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** Compaction Stats [default] ** 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:57:03.080 INFO:journalctl@ceph.mon.c.smithi172.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:57:03.081 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:57:03.082 INFO:journalctl@ceph.mon.c.smithi172.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T01:57:03.082 INFO:journalctl@ceph.mon.c.smithi172.stdout: Block cache BinnedLRUCache@0x55d1429511f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-16T01:57:03.082 INFO:journalctl@ceph.mon.c.smithi172.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T01:57:03.082 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-09-16T01:57:03.082 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T01:57:03.082 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-16T01:57:03.083 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: using public_addr v2:172.21.15.172:0/0 -> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] 2024-09-16T01:57:03.083 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: starting mon.c rank -1 at public addrs [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] at bind addrs [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:03.083 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(???) e0 preinit fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:03.083 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).mds e1 new map 2024-09-16T01:57:03.083 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-16T01:57:03.083 INFO:journalctl@ceph.mon.c.smithi172.stdout: e1 2024-09-16T01:57:03.083 INFO:journalctl@ceph.mon.c.smithi172.stdout: btime 2024-09-16T01:55:37:666788+0000 2024-09-16T01:57:03.083 INFO:journalctl@ceph.mon.c.smithi172.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T01:57:03.084 INFO:journalctl@ceph.mon.c.smithi172.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T01:57:03.084 INFO:journalctl@ceph.mon.c.smithi172.stdout: legacy client fscid: -1 2024-09-16T01:57:03.084 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-09-16T01:57:03.084 INFO:journalctl@ceph.mon.c.smithi172.stdout: No filesystems configured 2024-09-16T01:57:03.084 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T01:57:03.085 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T01:57:03.085 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T01:57:03.085 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T01:57:03.085 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T01:57:03.085 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T01:57:03.085 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T01:57:03.085 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:03.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Deploying cephadm binary to smithi172 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Updating smithi136:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Updating smithi136:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:03.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Added host smithi172 2024-09-16T01:57:03.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:03.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:57:03.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1819527321' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T01:57:03.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:03.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1819527321' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T01:57:03.089 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:03.089 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.089 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.089 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.089 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.089 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:03.089 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:03.089 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:03.090 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-16T01:57:03.090 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Updating smithi172:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:03.090 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:57:03.090 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Updating smithi172:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:57:03.090 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.090 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.090 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi136:172.21.15.136=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi136:172.21.15.136=b;smithi172:172.21.15.172=c;count:3 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: Deploying daemon mon.c on smithi172 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/2856748963' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:03.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 ceph-mon[35340]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T01:57:03.092 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 bash[35321]: 9e5b8f07d99b33ea7b16072cc45426c2d39c19a7bd488b370303ada7e7a2bb20 2024-09-16T01:57:03.092 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:02 smithi172 systemd[1]: Started Ceph mon.c for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:57:03.301 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:57:03.301 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:57:03.552 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:57:08.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:57:08.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: mon.a calling monitor election 2024-09-16T01:57:08.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:08.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: mon.c calling monitor election 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: monmap epoch 2 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: last_changed 2024-09-16T01:57:02.749283+0000 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: min_mon_release 19 (squid) 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: election_strategy: 1 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:57:08.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-16T01:57:08.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: fsmap 2024-09-16T01:57:08.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:08.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: mgrmap e13: x(active, since 39s) 2024-09-16T01:57:08.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: overall HEALTH_OK 2024-09-16T01:57:08.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:08.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:08.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:08.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:08.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:08.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: mon.a calling monitor election 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: mon.c calling monitor election 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:57:08.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: monmap epoch 2 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: last_changed 2024-09-16T01:57:02.749283+0000 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: min_mon_release 19 (squid) 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: election_strategy: 1 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: fsmap 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: mgrmap e13: x(active, since 39s) 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: overall HEALTH_OK 2024-09-16T01:57:08.297 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:08.298 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:08.298 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:08.298 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:08.298 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:08.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.354+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1333220487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba80003530 msgr2=0x7fba800059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:08.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.354+0000 7fba9d0b6640 1 --2- 172.21.15.172:0/1333220487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba80003530 0x7fba800059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fba981016a0 tx=0x7fba8802c490 comp rx=0 tx=0).stop 2024-09-16T01:57:08.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.354+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1333220487 shutdown_connections 2024-09-16T01:57:08.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.354+0000 7fba9d0b6640 1 --2- 172.21.15.172:0/1333220487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba80003530 0x7fba800059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.354+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1333220487 >> 172.21.15.172:0/1333220487 conn(0x7fba980fc220 msgr2=0x7fba980fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.354+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1333220487 shutdown_connections 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.354+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1333220487 wait complete. 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.355+0000 7fba9d0b6640 1 Processor -- start 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba9d0b6640 1 -- start start 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba9d0b6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba80003530 0x7fba98196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba9d0b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba981966d0 0x7fba9819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba9d0b6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba98108670 con 0x7fba981966d0 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba9d0b6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fba981084f0 con 0x7fba80003530 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba96d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba80003530 0x7fba98196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba96d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba80003530 0x7fba98196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45240/0 (socket says 172.21.15.172:45240) 2024-09-16T01:57:08.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba96d9d640 1 -- 172.21.15.172:0/1892037863 learned_addr learned my addr 172.21.15.172:0/1892037863 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T01:57:08.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba96d9d640 1 -- 172.21.15.172:0/1892037863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba80003530 msgr2=0x7fba98196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T01:57:08.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba96d9d640 1 -- 172.21.15.172:0/1892037863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba80003530 msgr2=0x7fba98196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:57:08.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba9659c640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba981966d0 0x7fba9819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:08.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.356+0000 7fba96d9d640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba80003530 0x7fba98196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:57:08.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.357+0000 7fba96d9d640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba80003530 0x7fba98196190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:57:08.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.357+0000 7fba9659c640 1 -- 172.21.15.172:0/1892037863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba80003530 msgr2=0x7fba98196190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:57:08.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.357+0000 7fba9659c640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba80003530 0x7fba98196190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.357+0000 7fba9659c640 1 -- 172.21.15.172:0/1892037863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba98196cf0 con 0x7fba981966d0 2024-09-16T01:57:08.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.357+0000 7fba9659c640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba981966d0 0x7fba9819aa60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00b3d0 tx=0x7fba8c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:08.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.357+0000 7fba7ffff640 1 -- 172.21.15.172:0/1892037863 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fba8c011630 con 0x7fba981966d0 2024-09-16T01:57:08.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.358+0000 7fba7ffff640 1 -- 172.21.15.172:0/1892037863 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba8c011cb0 con 0x7fba981966d0 2024-09-16T01:57:08.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.358+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1892037863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9819afa0 con 0x7fba981966d0 2024-09-16T01:57:08.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.358+0000 7fba7ffff640 1 -- 172.21.15.172:0/1892037863 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fba8c010df0 con 0x7fba981966d0 2024-09-16T01:57:08.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.358+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1892037863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9819b560 con 0x7fba981966d0 2024-09-16T01:57:08.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.359+0000 7fba7ffff640 1 -- 172.21.15.172:0/1892037863 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fba8c018020 con 0x7fba981966d0 2024-09-16T01:57:08.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.359+0000 7fba7ffff640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba84039090 0x7fba8403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:08.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.359+0000 7fba7ffff640 1 -- 172.21.15.172:0/1892037863 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fba8c04c0c0 con 0x7fba981966d0 2024-09-16T01:57:08.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.359+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1892037863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9819b870 con 0x7fba981966d0 2024-09-16T01:57:08.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.360+0000 7fba96d9d640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba84039090 0x7fba8403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:08.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.360+0000 7fba96d9d640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba84039090 0x7fba8403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fba88012c10 tx=0x7fba880323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:08.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.365+0000 7fba7ffff640 1 -- 172.21.15.172:0/1892037863 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba8c01f020 con 0x7fba981966d0 2024-09-16T01:57:08.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.599+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1892037863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fba9818fe50 con 0x7fba981966d0 2024-09-16T01:57:08.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.599+0000 7fba7ffff640 1 -- 172.21.15.172:0/1892037863 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fba8c018310 con 0x7fba981966d0 2024-09-16T01:57:08.601 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:57:08.601 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":2,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","modified":"2024-09-16T01:57:02.749283Z","created":"2024-09-16T01:55:32.719520Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T01:57:08.601 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 2 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba84039090 msgr2=0x7fba8403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba84039090 0x7fba8403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fba88012c10 tx=0x7fba880323a0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba981966d0 msgr2=0x7fba9819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba981966d0 0x7fba9819aa60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00b3d0 tx=0x7fba8c00b890 comp rx=0 tx=0).stop 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1892037863 shutdown_connections 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba84039090 0x7fba8403b550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba981966d0 0x7fba9819aa60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 --2- 172.21.15.172:0/1892037863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba80003530 0x7fba98196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1892037863 >> 172.21.15.172:0/1892037863 conn(0x7fba980fc220 msgr2=0x7fba980fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1892037863 shutdown_connections 2024-09-16T01:57:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:08.601+0000 7fba9d0b6640 1 -- 172.21.15.172:0/1892037863 wait complete. 2024-09-16T01:57:08.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:08 smithi172 ceph-mon[35340]: Deploying daemon mon.b on smithi136 2024-09-16T01:57:08.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:08 smithi172 ceph-mon[35340]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:08.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:08 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/1892037863' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:08.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:08 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:09.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:08 smithi103 ceph-mon[30398]: Deploying daemon mon.b on smithi136 2024-09-16T01:57:09.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:08 smithi103 ceph-mon[30398]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:09.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:08 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/1892037863' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:09.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:08 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:10.005 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 01:57:09 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T01:57:09.748+0000 7fce70e00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T01:57:10.235 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:57:10.235 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:57:10.490 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:57:10.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.862+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1005740530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe368107660 msgr2=0x7fe368107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:10.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.862+0000 7fe36f1d2640 1 --2- 172.21.15.172:0/1005740530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe368107660 0x7fe368107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009880 tx=0x7fe35c02a5d0 comp rx=0 tx=0).stop 2024-09-16T01:57:10.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.862+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1005740530 shutdown_connections 2024-09-16T01:57:10.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.862+0000 7fe36f1d2640 1 --2- 172.21.15.172:0/1005740530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe368107660 0x7fe368107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:10.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.862+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1005740530 >> 172.21.15.172:0/1005740530 conn(0x7fe3680fd320 msgr2=0x7fe3680ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:10.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.863+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1005740530 shutdown_connections 2024-09-16T01:57:10.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.863+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1005740530 wait complete. 2024-09-16T01:57:10.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.863+0000 7fe36f1d2640 1 Processor -- start 2024-09-16T01:57:10.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.863+0000 7fe36f1d2640 1 -- start start 2024-09-16T01:57:10.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.863+0000 7fe36f1d2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe368107660 0x7fe3681995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:10.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.863+0000 7fe36f1d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe368199b20 0x7fe36819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:10.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe36f1d2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe36810bd20 con 0x7fe368107660 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe36f1d2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe36810bba0 con 0x7fe368199b20 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe367fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe368199b20 0x7fe36819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe367fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe368199b20 0x7fe36819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45242/0 (socket says 172.21.15.172:45242) 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe367fff640 1 -- 172.21.15.172:0/1511920494 learned_addr learned my addr 172.21.15.172:0/1511920494 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe367fff640 1 -- 172.21.15.172:0/1511920494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe368199b20 msgr2=0x7fe36819deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe367fff640 1 -- 172.21.15.172:0/1511920494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe368199b20 msgr2=0x7fe36819deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe367fff640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe368199b20 0x7fe36819deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe36cf6e640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe368107660 0x7fe3681995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe367fff640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe368199b20 0x7fe36819deb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe36cf6e640 1 -- 172.21.15.172:0/1511920494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe368199b20 msgr2=0x7fe36819deb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe36cf6e640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe368199b20 0x7fe36819deb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe36cf6e640 1 -- 172.21.15.172:0/1511920494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36819a140 con 0x7fe368107660 2024-09-16T01:57:10.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.864+0000 7fe36cf6e640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe368107660 0x7fe3681995e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe35c005ab0 tx=0x7fe35c032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:10.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.865+0000 7fe365ffb640 1 -- 172.21.15.172:0/1511920494 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe35c035070 con 0x7fe368107660 2024-09-16T01:57:10.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.865+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1511920494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36819e3f0 con 0x7fe368107660 2024-09-16T01:57:10.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.865+0000 7fe365ffb640 1 -- 172.21.15.172:0/1511920494 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe35c032e40 con 0x7fe368107660 2024-09-16T01:57:10.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.865+0000 7fe365ffb640 1 -- 172.21.15.172:0/1511920494 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe35c03bc10 con 0x7fe368107660 2024-09-16T01:57:10.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.865+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1511920494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36819e930 con 0x7fe368107660 2024-09-16T01:57:10.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.866+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1511920494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe368102da0 con 0x7fe368107660 2024-09-16T01:57:10.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.866+0000 7fe365ffb640 1 -- 172.21.15.172:0/1511920494 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fe35c042030 con 0x7fe368107660 2024-09-16T01:57:10.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.866+0000 7fe365ffb640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe358039040 0x7fe35803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:10.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.866+0000 7fe365ffb640 1 -- 172.21.15.172:0/1511920494 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe35c040070 con 0x7fe368107660 2024-09-16T01:57:10.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.867+0000 7fe367fff640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe358039040 0x7fe35803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:10.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.867+0000 7fe367fff640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe358039040 0x7fe35803b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe354009100 tx=0x7fe354008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:10.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:10.872+0000 7fe365ffb640 1 -- 172.21.15.172:0/1511920494 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe35c03bdb0 con 0x7fe368107660 2024-09-16T01:57:11.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.100+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1511920494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe36819ec10 con 0x7fe368107660 2024-09-16T01:57:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.101+0000 7fe365ffb640 1 -- 172.21.15.172:0/1511920494 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fe35c03b460 con 0x7fe368107660 2024-09-16T01:57:11.102 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:57:11.102 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":2,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","modified":"2024-09-16T01:57:02.749283Z","created":"2024-09-16T01:55:32.719520Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T01:57:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 2 2024-09-16T01:57:11.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe358039040 msgr2=0x7fe35803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:11.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe358039040 0x7fe35803b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe354009100 tx=0x7fe354008040 comp rx=0 tx=0).stop 2024-09-16T01:57:11.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe368107660 msgr2=0x7fe3681995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:11.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe368107660 0x7fe3681995e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe35c005ab0 tx=0x7fe35c032c50 comp rx=0 tx=0).stop 2024-09-16T01:57:11.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1511920494 shutdown_connections 2024-09-16T01:57:11.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe358039040 0x7fe35803b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:11.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe368199b20 0x7fe36819deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:11.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 --2- 172.21.15.172:0/1511920494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe368107660 0x7fe3681995e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:11.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1511920494 >> 172.21.15.172:0/1511920494 conn(0x7fe3680fd320 msgr2=0x7fe368074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:11.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1511920494 shutdown_connections 2024-09-16T01:57:11.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:11.103+0000 7fe36f1d2640 1 -- 172.21.15.172:0/1511920494 wait complete. 2024-09-16T01:57:11.665 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T01:57:11.665 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.db_log_dir: 2024-09-16T01:57:11.665 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.wal_dir: 2024-09-16T01:57:11.666 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T01:57:11.666 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T01:57:11.666 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T01:57:11.666 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T01:57:11.666 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T01:57:11.666 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T01:57:11.666 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T01:57:11.666 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T01:57:11.666 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.write_buffer_manager: 0x5605d95f4dc0 2024-09-16T01:57:11.666 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T01:57:11.667 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T01:57:11.667 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T01:57:11.667 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T01:57:11.667 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T01:57:11.667 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T01:57:11.667 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T01:57:11.667 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T01:57:11.667 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.unordered_write: 0 2024-09-16T01:57:11.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:11 smithi172 ceph-mon[35340]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:11.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:11 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/1511920494' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:11.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:11 smithi103 ceph-mon[30398]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:11.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:11 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/1511920494' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:12.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T01:57:12.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.row_cache: None 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.wal_filter: None 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.two_write_queues: 0 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.wal_compression: 0 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.atomic_flush: 0 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T01:57:12.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.log_readahead_size: 0 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_background_jobs: 2 2024-09-16T01:57:12.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_background_compactions: -1 2024-09-16T01:57:12.028 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_subcompactions: 1 2024-09-16T01:57:12.028 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_open_files: -1 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T01:57:12.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_background_flushes: -1 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Compression algorithms supported: 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: kZSTD supported: 0 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: kXpressCompression supported: 0 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: kBZip2Compression supported: 0 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: kLZ4Compression supported: 1 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: kZlibCompression supported: 1 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: kSnappyCompression supported: 1 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-16T01:57:12.030 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.merge_operator: 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_filter: None 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5605d95e5320) 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout: cache_index_and_filter_blocks: 1 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout: pin_top_level_index_and_filter: 1 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout: index_type: 0 2024-09-16T01:57:12.031 INFO:journalctl@ceph.mon.b.smithi136.stdout: data_block_index_type: 0 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: index_shortening: 1 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: checksum: 4 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: no_block_cache: 0 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_cache: 0x5605d95d91f0 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_cache_name: BinnedLRUCache 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_cache_options: 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: capacity : 536870912 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: num_shard_bits : 4 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: strict_capacity_limit : 0 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: high_pri_pool_ratio: 0.000 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_cache_compressed: (nil) 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: persistent_cache: (nil) 2024-09-16T01:57:12.032 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_size: 4096 2024-09-16T01:57:12.033 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_size_deviation: 10 2024-09-16T01:57:12.033 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_restart_interval: 16 2024-09-16T01:57:12.033 INFO:journalctl@ceph.mon.b.smithi136.stdout: index_block_restart_interval: 1 2024-09-16T01:57:12.033 INFO:journalctl@ceph.mon.b.smithi136.stdout: metadata_block_size: 4096 2024-09-16T01:57:12.033 INFO:journalctl@ceph.mon.b.smithi136.stdout: partition_filters: 0 2024-09-16T01:57:12.033 INFO:journalctl@ceph.mon.b.smithi136.stdout: use_delta_encoding: 1 2024-09-16T01:57:12.033 INFO:journalctl@ceph.mon.b.smithi136.stdout: filter_policy: bloomfilter 2024-09-16T01:57:12.034 INFO:journalctl@ceph.mon.b.smithi136.stdout: whole_key_filtering: 1 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout: verify_compression: 0 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout: read_amp_bytes_per_bit: 0 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout: format_version: 5 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout: enable_index_compression: 1 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_align: 0 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout: max_auto_readahead_size: 262144 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout: prepopulate_block_cache: 0 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout: initial_auto_readahead_size: 8192 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compression: NoCompression 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T01:57:12.035 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.num_levels: 7 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T01:57:12.036 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T01:57:12.037 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T01:57:12.038 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T01:57:12.039 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.inplace_update_support: 0 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.bloom_locality: 0 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.max_successive_merges: 0 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T01:57:12.040 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.ttl: 2592000 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.enable_blob_files: false 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.min_blob_size: 0 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T01:57:12.041 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 94cf8e5c-e3e7-44f9-ae3f-8f4fea940275 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451831664168, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451831664669, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726451831, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "94cf8e5c-e3e7-44f9-ae3f-8f4fea940275", "db_session_id": "CJTED6N5DQ6I5CD4H69E", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726451831664773, "job": 1, "event": "recovery_finished"} 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5605d9606e00 2024-09-16T01:57:12.042 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: DB pointer 0x5605d9702000 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: ** DB Stats ** 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: ** Compaction Stats [default] ** 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:57:12.043 INFO:journalctl@ceph.mon.b.smithi136.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: ** Compaction Stats [default] ** 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:57:12.044 INFO:journalctl@ceph.mon.b.smithi136.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: Block cache BinnedLRUCache@0x5605d95d91f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-16T01:57:12.045 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: using public_addr v2:172.21.15.136:0/0 -> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: starting mon.b rank -1 at public addrs [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] at bind addrs [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(???) e0 preinit fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).mds e1 new map 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout: e1 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout: btime 2024-09-16T01:55:37:666788+0000 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout: legacy client fscid: -1 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout: No filesystems configured 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T01:57:12.046 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.047 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.048 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:12.048 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:12.048 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:12.048 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-16T01:57:12.048 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.048 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Deploying cephadm binary to smithi172 2024-09-16T01:57:12.048 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Updating smithi136:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:12.048 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.048 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Updating smithi136:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Added host smithi172 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1819527321' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T01:57:12.049 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1819527321' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Updating smithi172:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:57:12.050 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Updating smithi172:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi136:172.21.15.136=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi136:172.21.15.136=b;smithi172:172.21.15.172=c;count:3 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Deploying daemon mon.c on smithi172 2024-09-16T01:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/2856748963' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.a calling monitor election 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.c calling monitor election 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: monmap epoch 2 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: last_changed 2024-09-16T01:57:02.749283+0000 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: min_mon_release 19 (squid) 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: election_strategy: 1 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: fsmap 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mgrmap e13: x(active, since 39s) 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: overall HEALTH_OK 2024-09-16T01:57:12.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: Deploying daemon mon.b on smithi136 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/1892037863' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/1511920494' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 ceph-mon[35323]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 bash[35304]: 66e9a9454b023ccd9af4643b53dbc95928df0f6c70e15eba56d8bb2798bf63d6 2024-09-16T01:57:12.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:11 smithi136 systemd[1]: Started Ceph mon.b for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:57:12.701 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:57:12.701 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:57:12.951 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:57:16.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.989+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/3271546033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb70003530 msgr2=0x7fdb700059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.989+0000 7fdb8e7a3640 1 --2- 172.21.15.172:0/3271546033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb70003530 0x7fdb700059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdb8806a9a0 tx=0x7fdb7802c3f0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.989+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/3271546033 shutdown_connections 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.989+0000 7fdb8e7a3640 1 --2- 172.21.15.172:0/3271546033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb70003530 0x7fdb700059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.989+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/3271546033 >> 172.21.15.172:0/3271546033 conn(0x7fdb880fc220 msgr2=0x7fdb880fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.989+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/3271546033 shutdown_connections 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.989+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/3271546033 wait complete. 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.989+0000 7fdb8e7a3640 1 Processor -- start 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.989+0000 7fdb8e7a3640 1 -- start start 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb8e7a3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb70003530 0x7fdb881973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb8e7a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb881978f0 0x7fdb8819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb8e7a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb8819c1c0 0x7fdb8819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb8e7a3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb88109790 con 0x7fdb8819c1c0 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb8e7a3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb88109610 con 0x7fdb70003530 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb8e7a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb88109910 con 0x7fdb881978f0 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb877fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb881978f0 0x7fdb8819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb877fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb881978f0 0x7fdb8819bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57522/0 (socket says 172.21.15.172:57522) 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb877fe640 1 -- 172.21.15.172:0/2306465824 learned_addr learned my addr 172.21.15.172:0/2306465824 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb87fff640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb70003530 0x7fdb881973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb877fe640 1 -- 172.21.15.172:0/2306465824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb881978f0 msgr2=0x7fdb8819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb877fe640 1 -- 172.21.15.172:0/2306465824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb881978f0 msgr2=0x7fdb8819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb877fe640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb881978f0 0x7fdb8819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb87fff640 1 -- 172.21.15.172:0/2306465824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb70003530 msgr2=0x7fdb881973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb877fe640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb881978f0 0x7fdb8819bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb87fff640 1 -- 172.21.15.172:0/2306465824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb70003530 msgr2=0x7fdb881973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:57:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb87fff640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb70003530 0x7fdb881973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:57:16.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb87fff640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb70003530 0x7fdb881973b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:57:16.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.990+0000 7fdb8cd40640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb8819c1c0 0x7fdb8819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.991+0000 7fdb8cd40640 1 -- 172.21.15.172:0/2306465824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb881978f0 msgr2=0x7fdb8819bc80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:57:16.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.991+0000 7fdb8cd40640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb881978f0 0x7fdb8819bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.991+0000 7fdb8cd40640 1 -- 172.21.15.172:0/2306465824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb70003530 msgr2=0x7fdb881973b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:57:16.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.991+0000 7fdb8cd40640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb70003530 0x7fdb881973b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.991+0000 7fdb8cd40640 1 -- 172.21.15.172:0/2306465824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb881a3f30 con 0x7fdb8819c1c0 2024-09-16T01:57:16.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.991+0000 7fdb8cd40640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb8819c1c0 0x7fdb8819c670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000bd50 tx=0x7fdb8000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.992+0000 7fdb857fa640 1 -- 172.21.15.172:0/2306465824 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80019070 con 0x7fdb8819c1c0 2024-09-16T01:57:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.992+0000 7fdb857fa640 1 -- 172.21.15.172:0/2306465824 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb80014920 con 0x7fdb8819c1c0 2024-09-16T01:57:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.992+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/2306465824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb881a4160 con 0x7fdb8819c1c0 2024-09-16T01:57:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.992+0000 7fdb857fa640 1 -- 172.21.15.172:0/2306465824 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80012990 con 0x7fdb8819c1c0 2024-09-16T01:57:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.992+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/2306465824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb881a4670 con 0x7fdb8819c1c0 2024-09-16T01:57:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.993+0000 7fdb857fa640 1 -- 172.21.15.172:0/2306465824 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fdb8001c070 con 0x7fdb8819c1c0 2024-09-16T01:57:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.993+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/2306465824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8810a8a0 con 0x7fdb8819c1c0 2024-09-16T01:57:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.993+0000 7fdb857fa640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdb74039730 0x7fdb7403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.993+0000 7fdb857fa640 1 -- 172.21.15.172:0/2306465824 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdb8004d0d0 con 0x7fdb8819c1c0 2024-09-16T01:57:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.993+0000 7fdb87fff640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdb74039730 0x7fdb7403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.994+0000 7fdb87fff640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdb74039730 0x7fdb7403bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdb78014cc0 tx=0x7fdb78005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:16.998+0000 7fdb857fa640 1 -- 172.21.15.172:0/2306465824 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb80011df0 con 0x7fdb8819c1c0 2024-09-16T01:57:17.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:57:17.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: mon.a calling monitor election 2024-09-16T01:57:17.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: mon.c calling monitor election 2024-09-16T01:57:17.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:17.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:17.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:17.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:17.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: monmap epoch 3 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: last_changed 2024-09-16T01:57:11.690996+0000 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: min_mon_release 19 (squid) 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: election_strategy: 1 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: fsmap 2024-09-16T01:57:17.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:17.008 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: mgrmap e13: x(active, since 48s) 2024-09-16T01:57:17.008 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: overall HEALTH_OK 2024-09-16T01:57:17.008 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:17.008 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:17.008 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:17.008 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:17.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:57:17.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: mon.a calling monitor election 2024-09-16T01:57:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: mon.c calling monitor election 2024-09-16T01:57:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: monmap epoch 3 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: last_changed 2024-09-16T01:57:11.690996+0000 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: min_mon_release 19 (squid) 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: election_strategy: 1 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: fsmap 2024-09-16T01:57:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:17.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: mgrmap e13: x(active, since 48s) 2024-09-16T01:57:17.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: overall HEALTH_OK 2024-09-16T01:57:17.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:17.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:17.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:17.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:17.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.243+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/2306465824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fdb881a4a60 con 0x7fdb8819c1c0 2024-09-16T01:57:17.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.244+0000 7fdb857fa640 1 -- 172.21.15.172:0/2306465824 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fdb8000e090 con 0x7fdb8819c1c0 2024-09-16T01:57:17.246 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:57:17.246 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":3,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","modified":"2024-09-16T01:57:11.690996Z","created":"2024-09-16T01:55:32.719520Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T01:57:17.246 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 3 2024-09-16T01:57:17.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdb74039730 msgr2=0x7fdb7403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdb74039730 0x7fdb7403bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdb78014cc0 tx=0x7fdb78005f60 comp rx=0 tx=0).stop 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb8819c1c0 msgr2=0x7fdb8819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb8819c1c0 0x7fdb8819c670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000bd50 tx=0x7fdb8000be40 comp rx=0 tx=0).stop 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/2306465824 shutdown_connections 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdb74039730 0x7fdb7403bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb8819c1c0 0x7fdb8819c670 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb881978f0 0x7fdb8819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 --2- 172.21.15.172:0/2306465824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb70003530 0x7fdb881973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/2306465824 >> 172.21.15.172:0/2306465824 conn(0x7fdb880fc220 msgr2=0x7fdb880fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/2306465824 shutdown_connections 2024-09-16T01:57:17.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:17.247+0000 7fdb8e7a3640 1 -- 172.21.15.172:0/2306465824 wait complete. 2024-09-16T01:57:17.830 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:17.830 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:17 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/2306465824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:17.830 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:17.832 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T01:57:17.832 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T01:57:18.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:18.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:17 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/2306465824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:18.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:18.203 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:57:18.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.777+0000 7fc2d2600640 1 -- 172.21.15.103:0/1979809457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2cc109650 msgr2=0x7fc2cc109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:18.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.777+0000 7fc2d2600640 1 --2- 172.21.15.103:0/1979809457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2cc109650 0x7fc2cc109a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009880 tx=0x7fc2c002a5d0 comp rx=0 tx=0).stop 2024-09-16T01:57:18.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.778+0000 7fc2d2600640 1 -- 172.21.15.103:0/1979809457 shutdown_connections 2024-09-16T01:57:18.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.778+0000 7fc2d2600640 1 --2- 172.21.15.103:0/1979809457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2cc109650 0x7fc2cc109a30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:18.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.778+0000 7fc2d2600640 1 -- 172.21.15.103:0/1979809457 >> 172.21.15.103:0/1979809457 conn(0x7fc2cc0ff260 msgr2=0x7fc2cc101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:18.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.778+0000 7fc2d2600640 1 -- 172.21.15.103:0/1979809457 shutdown_connections 2024-09-16T01:57:18.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.778+0000 7fc2d2600640 1 -- 172.21.15.103:0/1979809457 wait complete. 2024-09-16T01:57:18.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.779+0000 7fc2d2600640 1 Processor -- start 2024-09-16T01:57:18.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.779+0000 7fc2d2600640 1 -- start start 2024-09-16T01:57:18.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.779+0000 7fc2d2600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2cc109650 0x7fc2cc076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.779+0000 7fc2d2600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2cc07a660 0x7fc2cc076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.779+0000 7fc2d2600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0772e0 0x7fc2cc1a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.779+0000 7fc2d2600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc10dc60 con 0x7fc2cc109650 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.779+0000 7fc2d2600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc10dae0 con 0x7fc2cc07a660 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.779+0000 7fc2d2600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc10dde0 con 0x7fc2cc0772e0 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.780+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2cc07a660 0x7fc2cc076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.780+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2cc07a660 0x7fc2cc076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:54556/0 (socket says 172.21.15.103:54556) 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.780+0000 7fc2cbe00640 1 -- 172.21.15.103:0/2378610558 learned_addr learned my addr 172.21.15.103:0/2378610558 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.780+0000 7fc2d1c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0772e0 0x7fc2cc1a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.780+0000 7fc2d1200640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2cc109650 0x7fc2cc076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.780+0000 7fc2cbe00640 1 -- 172.21.15.103:0/2378610558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0772e0 msgr2=0x7fc2cc1a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.780+0000 7fc2cbe00640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0772e0 0x7fc2cc1a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.780+0000 7fc2cbe00640 1 -- 172.21.15.103:0/2378610558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2cc109650 msgr2=0x7fc2cc076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.780+0000 7fc2cbe00640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2cc109650 0x7fc2cc076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.780+0000 7fc2cbe00640 1 -- 172.21.15.103:0/2378610558 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc1aa160 con 0x7fc2cc07a660 2024-09-16T01:57:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.781+0000 7fc2cbe00640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2cc07a660 0x7fc2cc076c40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800b520 tx=0x7fc2b800b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.781+0000 7fc2c9600640 1 -- 172.21.15.103:0/2378610558 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8004230 con 0x7fc2cc07a660 2024-09-16T01:57:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.781+0000 7fc2c9600640 1 -- 172.21.15.103:0/2378610558 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2b80043d0 con 0x7fc2cc07a660 2024-09-16T01:57:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.781+0000 7fc2c9600640 1 -- 172.21.15.103:0/2378610558 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8003d60 con 0x7fc2cc07a660 2024-09-16T01:57:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.781+0000 7fc2d2600640 1 -- 172.21.15.103:0/2378610558 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc1aa450 con 0x7fc2cc07a660 2024-09-16T01:57:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.781+0000 7fc2d2600640 1 -- 172.21.15.103:0/2378610558 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc1aaa10 con 0x7fc2cc07a660 2024-09-16T01:57:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.782+0000 7fc2c9600640 1 -- 172.21.15.103:0/2378610558 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fc2b800e070 con 0x7fc2cc07a660 2024-09-16T01:57:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.783+0000 7fc2d2600640 1 -- 172.21.15.103:0/2378610558 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2cc104d00 con 0x7fc2cc07a660 2024-09-16T01:57:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.783+0000 7fc2c9600640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc2b4039730 0x7fc2b403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.783+0000 7fc2c9600640 1 -- 172.21.15.103:0/2378610558 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc2b804be00 con 0x7fc2cc07a660 2024-09-16T01:57:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.783+0000 7fc2d1200640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc2b4039730 0x7fc2b403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.784+0000 7fc2d1200640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc2b4039730 0x7fc2b403bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0000c00 tx=0x7fc2c0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:18.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:18.788+0000 7fc2c9600640 1 -- 172.21.15.103:0/2378610558 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2b8014020 con 0x7fc2cc07a660 2024-09-16T01:57:19.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.017+0000 7fc2d2600640 1 -- 172.21.15.103:0/2378610558 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc2cc1aadd0 con 0x7fc2cc07a660 2024-09-16T01:57:19.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.018+0000 7fc2c9600640 1 -- 172.21.15.103:0/2378610558 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fc2b8020350 con 0x7fc2cc07a660 2024-09-16T01:57:19.021 INFO:teuthology.orchestra.run.smithi103.stdout:# minimal ceph.conf for a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:19.021 INFO:teuthology.orchestra.run.smithi103.stdout:[global] 2024-09-16T01:57:19.021 INFO:teuthology.orchestra.run.smithi103.stdout: fsid = a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:19.021 INFO:teuthology.orchestra.run.smithi103.stdout: mon_host = [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.020+0000 7fc2d2600640 1 -- 172.21.15.103:0/2378610558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc2b4039730 msgr2=0x7fc2b403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.020+0000 7fc2d2600640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc2b4039730 0x7fc2b403bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0000c00 tx=0x7fc2c0005a20 comp rx=0 tx=0).stop 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.020+0000 7fc2d2600640 1 -- 172.21.15.103:0/2378610558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2cc07a660 msgr2=0x7fc2cc076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.020+0000 7fc2d2600640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2cc07a660 0x7fc2cc076c40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800b520 tx=0x7fc2b800b9e0 comp rx=0 tx=0).stop 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.020+0000 7fc2d2600640 1 -- 172.21.15.103:0/2378610558 shutdown_connections 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.020+0000 7fc2d2600640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc2b4039730 0x7fc2b403bbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.020+0000 7fc2d2600640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0772e0 0x7fc2cc1a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.020+0000 7fc2d2600640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2cc07a660 0x7fc2cc076c40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.020+0000 7fc2d2600640 1 --2- 172.21.15.103:0/2378610558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2cc109650 0x7fc2cc076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.021+0000 7fc2d2600640 1 -- 172.21.15.103:0/2378610558 >> 172.21.15.103:0/2378610558 conn(0x7fc2cc0ff260 msgr2=0x7fc2cc0ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:19.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.021+0000 7fc2d2600640 1 -- 172.21.15.103:0/2378610558 shutdown_connections 2024-09-16T01:57:19.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:19.021+0000 7fc2d2600640 1 -- 172.21.15.103:0/2378610558 wait complete. 2024-09-16T01:57:19.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:57:19.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:19.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: mon.a calling monitor election 2024-09-16T01:57:19.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: mon.c calling monitor election 2024-09-16T01:57:19.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:57:19.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:19.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:19.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:19.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:19.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:19.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:19.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:19.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:19.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: monmap epoch 3 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: last_changed 2024-09-16T01:57:11.690996+0000 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: min_mon_release 19 (squid) 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: election_strategy: 1 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: fsmap 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:19.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: mgrmap e13: x(active, since 48s) 2024-09-16T01:57:19.027 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: overall HEALTH_OK 2024-09-16T01:57:19.027 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.027 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.027 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.027 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.027 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:19.027 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/2306465824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:57:19.027 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: mon.b calling monitor election 2024-09-16T01:57:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-16T01:57:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-16T01:57:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-16T01:57:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: mon.b calling monitor election 2024-09-16T01:57:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: mon.c calling monitor election 2024-09-16T01:57:19.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: mon.a calling monitor election 2024-09-16T01:57:19.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T01:57:19.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: monmap epoch 3 2024-09-16T01:57:19.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:19.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: last_changed 2024-09-16T01:57:11.690996+0000 2024-09-16T01:57:19.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:57:19.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: min_mon_release 19 (squid) 2024-09-16T01:57:19.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: election_strategy: 1 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: fsmap 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: mgrmap e13: x(active, since 50s) 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: overall HEALTH_OK 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: Updating smithi172:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: Updating smithi136:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2378610558' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:19.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.758 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.758 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.758 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.758 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:19 smithi103 ceph-mon[30398]: Updating smithi103:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:19.814 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T01:57:19.815 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:57:19.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T01:57:19.872 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:57:19.873 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:57:19.969 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:57:19.969 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T01:57:19.995 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: mon.b calling monitor election 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: mon.b calling monitor election 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: mon.c calling monitor election 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: mon.a calling monitor election 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: monmap epoch 3 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: last_changed 2024-09-16T01:57:11.690996+0000 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:57:19.996 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: min_mon_release 19 (squid) 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: election_strategy: 1 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: fsmap 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: mgrmap e13: x(active, since 50s) 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: overall HEALTH_OK 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: Updating smithi172:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: Updating smithi136:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2378610558' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.997 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.998 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.998 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:19.998 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:19 smithi136 ceph-mon[35323]: Updating smithi103:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:20.000 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:57:20.000 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:57:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: mon.b calling monitor election 2024-09-16T01:57:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-16T01:57:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-16T01:57:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-16T01:57:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: mon.b calling monitor election 2024-09-16T01:57:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: mon.c calling monitor election 2024-09-16T01:57:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: mon.a calling monitor election 2024-09-16T01:57:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: monmap epoch 3 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: last_changed 2024-09-16T01:57:11.690996+0000 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: created 2024-09-16T01:55:32.719520+0000 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: min_mon_release 19 (squid) 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: election_strategy: 1 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: fsmap 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: mgrmap e13: x(active, since 50s) 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: overall HEALTH_OK 2024-09-16T01:57:20.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: Updating smithi172:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:20.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: Updating smithi136:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:20.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2378610558' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:20.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:19 smithi172 ceph-mon[35340]: Updating smithi103:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:57:20.067 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:57:20.067 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T01:57:20.095 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:57:20.096 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:57:20.161 INFO:tasks.cephadm:Adding mgr.x on smithi103 2024-09-16T01:57:20.162 INFO:tasks.cephadm:Adding mgr.y on smithi136 2024-09-16T01:57:20.162 INFO:tasks.cephadm:Adding mgr.z on smithi172 2024-09-16T01:57:20.162 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi103=x;smithi136=y;smithi172=z' 2024-09-16T01:57:20.436 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:57:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T01:57:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:57:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: Reconfiguring daemon mon.a on smithi103 2024-09-16T01:57:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:20 smithi136 ceph-mon[35323]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:20.850 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.851 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.851 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.851 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:20.851 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.851 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.852 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.852 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.852 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T01:57:20.852 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:20.852 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:57:20.852 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:20.853 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: Reconfiguring daemon mon.a on smithi103 2024-09-16T01:57:20.853 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:20 smithi172 ceph-mon[35340]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:20.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.866+0000 7f0d1db6c640 1 -- 172.21.15.172:0/26598289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d181005f0 msgr2=0x7f0d181009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:20.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.866+0000 7f0d1db6c640 1 --2- 172.21.15.172:0/26598289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d181005f0 0x7f0d181009f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009880 tx=0x7f0d0802a5d0 comp rx=0 tx=0).stop 2024-09-16T01:57:20.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.866+0000 7f0d1db6c640 1 -- 172.21.15.172:0/26598289 shutdown_connections 2024-09-16T01:57:20.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.866+0000 7f0d1db6c640 1 --2- 172.21.15.172:0/26598289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d181005f0 0x7f0d181009f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:20.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.866+0000 7f0d1db6c640 1 -- 172.21.15.172:0/26598289 >> 172.21.15.172:0/26598289 conn(0x7f0d180fbda0 msgr2=0x7f0d180fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:20.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.866+0000 7f0d1db6c640 1 -- 172.21.15.172:0/26598289 shutdown_connections 2024-09-16T01:57:20.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.866+0000 7f0d1db6c640 1 -- 172.21.15.172:0/26598289 wait complete. 2024-09-16T01:57:20.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d1db6c640 1 Processor -- start 2024-09-16T01:57:20.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d1db6c640 1 -- start start 2024-09-16T01:57:20.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d1db6c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d181005f0 0x7f0d18197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:20.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d1db6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d18197560 0x7f0d1819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:20.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d1db6c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d181979e0 0x7f0d1819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:20.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d1db6c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d181066e0 con 0x7f0d18197560 2024-09-16T01:57:20.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d1db6c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d18106560 con 0x7f0d181005f0 2024-09-16T01:57:20.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d1db6c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d18106860 con 0x7f0d181979e0 2024-09-16T01:57:20.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d17fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d181979e0 0x7f0d1819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:20.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d17fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d181979e0 0x7f0d1819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57528/0 (socket says 172.21.15.172:57528) 2024-09-16T01:57:20.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d17fff640 1 -- 172.21.15.172:0/324714770 learned_addr learned my addr 172.21.15.172:0/324714770 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T01:57:20.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d177fe640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d181005f0 0x7f0d18197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:20.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d0ffff640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d18197560 0x7f0d1819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d17fff640 1 -- 172.21.15.172:0/324714770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d181005f0 msgr2=0x7f0d18197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d17fff640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d181005f0 0x7f0d18197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d17fff640 1 -- 172.21.15.172:0/324714770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d18197560 msgr2=0x7f0d1819c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d17fff640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d18197560 0x7f0d1819c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d17fff640 1 -- 172.21.15.172:0/324714770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1819f420 con 0x7f0d181979e0 2024-09-16T01:57:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.867+0000 7f0d17fff640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d181979e0 0x7f0d1819eb60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000ba50 tx=0x7f0d1000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.868+0000 7f0d157fa640 1 -- 172.21.15.172:0/324714770 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10019070 con 0x7f0d181979e0 2024-09-16T01:57:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.868+0000 7f0d157fa640 1 -- 172.21.15.172:0/324714770 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1000cbe0 con 0x7f0d181979e0 2024-09-16T01:57:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.868+0000 7f0d1db6c640 1 -- 172.21.15.172:0/324714770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1819f710 con 0x7f0d181979e0 2024-09-16T01:57:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.868+0000 7f0d157fa640 1 -- 172.21.15.172:0/324714770 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100076a0 con 0x7f0d181979e0 2024-09-16T01:57:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.868+0000 7f0d1db6c640 1 -- 172.21.15.172:0/324714770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1819fc80 con 0x7f0d181979e0 2024-09-16T01:57:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.869+0000 7f0d157fa640 1 -- 172.21.15.172:0/324714770 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f0d1001c070 con 0x7f0d181979e0 2024-09-16T01:57:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.869+0000 7f0d1db6c640 1 -- 172.21.15.172:0/324714770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1819ff40 con 0x7f0d181979e0 2024-09-16T01:57:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.869+0000 7f0d157fa640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d04039730 0x7f0d0403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.869+0000 7f0d157fa640 1 -- 172.21.15.172:0/324714770 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0d1004be80 con 0x7f0d181979e0 2024-09-16T01:57:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.869+0000 7f0d177fe640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d04039730 0x7f0d0403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.870+0000 7f0d177fe640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d04039730 0x7f0d0403bbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009230 tx=0x7f0d080315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:20.874+0000 7f0d157fa640 1 -- 172.21.15.172:0/324714770 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d1001d540 con 0x7f0d181979e0 2024-09-16T01:57:20.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:57:20.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:20.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T01:57:20.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:20.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:57:20.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:20.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: Reconfiguring daemon mon.a on smithi103 2024-09-16T01:57:20.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:20 smithi103 ceph-mon[30398]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:21.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.050+0000 7f0d1db6c640 1 -- 172.21.15.172:0/324714770 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi136=y;smithi172=z", "target": ["mon-mgr", ""]}) -- 0x7f0d18062630 con 0x7f0d04039730 2024-09-16T01:57:21.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.056+0000 7f0d157fa640 1 -- 172.21.15.172:0/324714770 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0d18062630 con 0x7f0d04039730 2024-09-16T01:57:21.057 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mgr update... 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 -- 172.21.15.172:0/324714770 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d04039730 msgr2=0x7f0d0403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d04039730 0x7f0d0403bbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009230 tx=0x7f0d080315d0 comp rx=0 tx=0).stop 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 -- 172.21.15.172:0/324714770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d181979e0 msgr2=0x7f0d1819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d181979e0 0x7f0d1819eb60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000ba50 tx=0x7f0d1000bf10 comp rx=0 tx=0).stop 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 -- 172.21.15.172:0/324714770 shutdown_connections 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d04039730 0x7f0d0403bbf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d181979e0 0x7f0d1819eb60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d18197560 0x7f0d1819c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 --2- 172.21.15.172:0/324714770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d181005f0 0x7f0d18197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 -- 172.21.15.172:0/324714770 >> 172.21.15.172:0/324714770 conn(0x7f0d180fbda0 msgr2=0x7f0d1806acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 -- 172.21.15.172:0/324714770 shutdown_connections 2024-09-16T01:57:21.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:57:21.058+0000 7f0d1db6c640 1 -- 172.21.15.172:0/324714770 wait complete. 2024-09-16T01:57:21.638 DEBUG:teuthology.orchestra.run.smithi136:mgr.y> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T01:57:21.641 DEBUG:teuthology.orchestra.run.smithi172:mgr.z> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.z.service 2024-09-16T01:57:21.643 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T01:57:21.643 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T01:57:21.643 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T01:57:21.679 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T01:57:21.679 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-09-16T01:57:21.755 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T01:57:21.755 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:21.756 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-09-16T01:57:21.756 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:21.756 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:21.756 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-16 01:56:41.088331768 +0000 2024-09-16T01:57:21.756 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-16 01:52:24.911565236 +0000 2024-09-16T01:57:21.756 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-16 01:52:24.911565236 +0000 2024-09-16T01:57:21.756 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-16 01:52:24.911565236 +0000 2024-09-16T01:57:21.756 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T01:57:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-16T01:57:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-16T01:57:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000440548 s, 1.2 MB/s 2024-09-16T01:57:21.875 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T01:57:21.912 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-09-16T01:57:21.988 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T01:57:21.988 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:21.988 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-09-16T01:57:21.988 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:21.988 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:21.988 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-16 01:56:41.140330809 +0000 2024-09-16T01:57:21.988 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-16 01:52:25.382556686 +0000 2024-09-16T01:57:21.989 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-16 01:52:25.382556686 +0000 2024-09-16T01:57:21.989 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-16 01:52:25.382556686 +0000 2024-09-16T01:57:21.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T01:57:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-16T01:57:22.085 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-16T01:57:22.085 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000291029 s, 1.8 MB/s 2024-09-16T01:57:22.087 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T01:57:22.125 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-09-16T01:57:22.202 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T01:57:22.202 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-09-16T01:57:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-16 01:56:41.190329887 +0000 2024-09-16T01:57:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-16 01:52:25.873547774 +0000 2024-09-16T01:57:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-16 01:52:25.873547774 +0000 2024-09-16T01:57:22.203 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-16 01:52:25.873547774 +0000 2024-09-16T01:57:22.203 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T01:57:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-16T01:57:22.296 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-16T01:57:22.296 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000267429 s, 1.9 MB/s 2024-09-16T01:57:22.298 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T01:57:22.334 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-09-16T01:57:22.411 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T01:57:22.411 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:22.411 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-09-16T01:57:22.412 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:22.412 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:22.412 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-16 01:56:41.243328910 +0000 2024-09-16T01:57:22.412 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-16 01:52:26.343539243 +0000 2024-09-16T01:57:22.412 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-16 01:52:26.343539243 +0000 2024-09-16T01:57:22.412 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-16 01:52:26.343539243 +0000 2024-09-16T01:57:22.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T01:57:22.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:22 smithi103 ceph-mon[30398]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi136=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:22.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:22 smithi103 ceph-mon[30398]: Saving service mgr spec with placement smithi103=x;smithi136=y;smithi172=z;count:3 2024-09-16T01:57:22.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:22.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:22.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:22.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:22 smithi103 ceph-mon[30398]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T01:57:22.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:57:22.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:57:22.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:22.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:22 smithi103 ceph-mon[30398]: Reconfiguring daemon mgr.x on smithi103 2024-09-16T01:57:22.507 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-16T01:57:22.508 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-16T01:57:22.508 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.00236306 s, 217 kB/s 2024-09-16T01:57:22.510 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T01:57:22.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:22 smithi136 ceph-mon[35323]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi136=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:22.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:22 smithi136 ceph-mon[35323]: Saving service mgr spec with placement smithi103=x;smithi136=y;smithi172=z;count:3 2024-09-16T01:57:22.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:22.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:22.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:22.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:22 smithi136 ceph-mon[35323]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T01:57:22.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:57:22.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:57:22.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:22.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:22 smithi136 ceph-mon[35323]: Reconfiguring daemon mgr.x on smithi103 2024-09-16T01:57:22.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:22 smithi172 ceph-mon[35340]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi136=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:22.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:22 smithi172 ceph-mon[35340]: Saving service mgr spec with placement smithi103=x;smithi136=y;smithi172=z;count:3 2024-09-16T01:57:22.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:22.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:22.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:22.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:22 smithi172 ceph-mon[35340]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T01:57:22.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:57:22.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:57:22.547 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:22.547 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:22 smithi172 ceph-mon[35340]: Reconfiguring daemon mgr.x on smithi103 2024-09-16T01:57:22.587 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T01:57:22.588 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T01:57:22.608 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T01:57:22.608 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-09-16T01:57:22.667 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T01:57:22.667 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:22.667 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T01:57:22.667 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:22.667 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:22.667 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-16 01:57:11.142782779 +0000 2024-09-16T01:57:22.668 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-16 01:51:50.457177144 +0000 2024-09-16T01:57:22.668 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-16 01:51:50.457177144 +0000 2024-09-16T01:57:22.668 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-16 01:51:50.457177144 +0000 2024-09-16T01:57:22.668 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T01:57:22.735 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-16T01:57:22.735 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-16T01:57:22.735 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000240229 s, 2.1 MB/s 2024-09-16T01:57:22.737 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T01:57:22.795 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-09-16T01:57:22.853 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T01:57:22.853 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:22.853 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T01:57:22.853 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:22.853 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:22.854 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-16 01:57:11.143782757 +0000 2024-09-16T01:57:22.854 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-16 01:51:50.795169371 +0000 2024-09-16T01:57:22.854 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-16 01:51:50.795169371 +0000 2024-09-16T01:57:22.854 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-16 01:51:50.795169371 +0000 2024-09-16T01:57:22.854 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T01:57:22.920 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-16T01:57:22.920 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-16T01:57:22.920 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000213366 s, 2.4 MB/s 2024-09-16T01:57:22.922 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T01:57:22.981 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-09-16T01:57:23.037 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T01:57:23.037 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:23.037 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-16T01:57:23.037 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:23.037 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:23.037 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-16 01:57:11.143782757 +0000 2024-09-16T01:57:23.037 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-16 01:51:51.122161852 +0000 2024-09-16T01:57:23.037 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-16 01:51:51.122161852 +0000 2024-09-16T01:57:23.038 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-16 01:51:51.122161852 +0000 2024-09-16T01:57:23.038 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T01:57:23.104 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-16T01:57:23.105 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-16T01:57:23.105 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000203846 s, 2.5 MB/s 2024-09-16T01:57:23.106 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T01:57:23.163 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-09-16T01:57:23.219 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T01:57:23.219 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:23.219 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-16T01:57:23.219 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:23.219 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:23.219 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-16 01:57:11.143782757 +0000 2024-09-16T01:57:23.219 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-16 01:51:51.476153711 +0000 2024-09-16T01:57:23.220 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-16 01:51:51.476153711 +0000 2024-09-16T01:57:23.220 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-16 01:51:51.476153711 +0000 2024-09-16T01:57:23.220 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T01:57:23.283 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-16T01:57:23.283 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-16T01:57:23.284 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000205144 s, 2.5 MB/s 2024-09-16T01:57:23.285 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T01:57:23.340 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-16T01:57:23.341 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T01:57:23.361 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T01:57:23.361 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-09-16T01:57:23.388 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:23 smithi136 ceph-mon[35323]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:23.388 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:23.388 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:23.388 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:23.388 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:57:23.388 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:23.419 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T01:57:23.419 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:23.420 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T01:57:23.420 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:23.420 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:23.420 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-16 01:57:01.799610572 +0000 2024-09-16T01:57:23.420 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-16 01:51:50.420160988 +0000 2024-09-16T01:57:23.420 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-16 01:51:50.420160988 +0000 2024-09-16T01:57:23.420 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-16 01:51:50.420160988 +0000 2024-09-16T01:57:23.420 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T01:57:23.484 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:23 smithi172 ceph-mon[35340]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:23.485 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:23.485 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:23.485 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:23.485 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:57:23.485 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-16T01:57:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-16T01:57:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.0002314 s, 2.2 MB/s 2024-09-16T01:57:23.489 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T01:57:23.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:23 smithi103 ceph-mon[30398]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:23.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:23.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:23.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:23.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:57:23.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:23.548 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-09-16T01:57:23.606 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T01:57:23.606 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:23.606 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-16T01:57:23.606 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:23.606 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:23.606 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-16 01:57:01.799610572 +0000 2024-09-16T01:57:23.606 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-16 01:51:50.740152849 +0000 2024-09-16T01:57:23.606 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-16 01:51:50.740152849 +0000 2024-09-16T01:57:23.607 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-16 01:51:50.740152849 +0000 2024-09-16T01:57:23.607 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T01:57:23.672 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-16T01:57:23.672 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-16T01:57:23.672 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000285707 s, 1.8 MB/s 2024-09-16T01:57:23.673 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T01:57:23.730 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-09-16T01:57:23.786 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T01:57:23.786 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:23.787 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-16T01:57:23.787 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:23.787 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:23.787 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-16 01:57:01.799610572 +0000 2024-09-16T01:57:23.787 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-16 01:51:51.065144582 +0000 2024-09-16T01:57:23.787 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-16 01:51:51.065144582 +0000 2024-09-16T01:57:23.787 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-16 01:51:51.065144582 +0000 2024-09-16T01:57:23.787 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T01:57:23.852 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-16T01:57:23.853 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-16T01:57:23.853 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000204776 s, 2.5 MB/s 2024-09-16T01:57:23.854 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T01:57:23.912 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-09-16T01:57:23.969 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T01:57:23.969 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:57:23.969 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T01:57:23.969 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:57:23.970 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:57:23.970 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-16 01:57:01.800610547 +0000 2024-09-16T01:57:23.970 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-16 01:51:51.398136112 +0000 2024-09-16T01:57:23.970 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-16 01:51:51.398136112 +0000 2024-09-16T01:57:23.970 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-16 01:51:51.398136112 +0000 2024-09-16T01:57:23.970 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T01:57:24.034 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-16T01:57:24.034 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-16T01:57:24.034 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000207201 s, 2.5 MB/s 2024-09-16T01:57:24.035 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T01:57:24.092 INFO:tasks.cephadm:Deploying osd.0 on smithi103 with /dev/vg_nvme/lv_4... 2024-09-16T01:57:24.092 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T01:57:24.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:24 smithi172 ceph-mon[35340]: Reconfiguring mon.b (monmap changed)... 2024-09-16T01:57:24.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:24 smithi172 ceph-mon[35340]: Reconfiguring daemon mon.b on smithi136 2024-09-16T01:57:24.464 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:57:24.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:24 smithi103 ceph-mon[30398]: Reconfiguring mon.b (monmap changed)... 2024-09-16T01:57:24.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:24 smithi103 ceph-mon[30398]: Reconfiguring daemon mon.b on smithi136 2024-09-16T01:57:24.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:24 smithi136 ceph-mon[35323]: Reconfiguring mon.b (monmap changed)... 2024-09-16T01:57:24.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:24 smithi136 ceph-mon[35323]: Reconfiguring daemon mon.b on smithi136 2024-09-16T01:57:25.459 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:25.459 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:25.459 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:25 smithi172 ceph-mon[35340]: Reconfiguring mon.c (monmap changed)... 2024-09-16T01:57:25.459 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:25.460 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:57:25.460 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:25.460 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:25 smithi172 ceph-mon[35340]: Reconfiguring daemon mon.c on smithi172 2024-09-16T01:57:25.460 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:25 smithi172 ceph-mon[35340]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:25.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:25.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:25.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:25 smithi103 ceph-mon[30398]: Reconfiguring mon.c (monmap changed)... 2024-09-16T01:57:25.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:25.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:57:25.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:25.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:25 smithi103 ceph-mon[30398]: Reconfiguring daemon mon.c on smithi172 2024-09-16T01:57:25.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:25 smithi103 ceph-mon[30398]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:25.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:25.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:25.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:25 smithi136 ceph-mon[35323]: Reconfiguring mon.c (monmap changed)... 2024-09-16T01:57:25.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:57:25.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:57:25.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:25.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:25 smithi136 ceph-mon[35323]: Reconfiguring daemon mon.c on smithi172 2024-09-16T01:57:25.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:25 smithi136 ceph-mon[35323]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:26.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:26.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:26.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:26.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:26.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:26.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:26.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:57:26.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:57:26.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:57:26.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:26.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:26 smithi103 ceph-mon[30398]: Deploying daemon mgr.z on smithi172 2024-09-16T01:57:26.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:26.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:26.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:26.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:26.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:26.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:26.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:57:26.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:57:26.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:57:26.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:26.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:26 smithi136 ceph-mon[35323]: Deploying daemon mgr.z on smithi172 2024-09-16T01:57:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:57:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:57:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:57:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:26.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:26 smithi172 ceph-mon[35340]: Deploying daemon mgr.z on smithi172 2024-09-16T01:57:27.252 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:57:27.282 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-09-16T01:57:27.644 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:57:27.747 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:27 smithi172 ceph-mon[35340]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:27.747 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:27 smithi172 systemd[1]: Started Ceph mgr.z for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T01:57:27.783 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:27 smithi103 ceph-mon[30398]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:27 smithi136 ceph-mon[35323]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:28.027 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:27 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:27.744+0000 7f0bf8f4b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:57:28.027 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:27 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:27.813+0000 7f0bf8f4b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:57:28.027 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:27 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:27.958+0000 7f0bf8f4b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:57:28.027 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:28.024+0000 7f0bf8f4b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:57:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.128+0000 7f6f29800640 1 -- 172.21.15.103:0/1645487729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24104f60 msgr2=0x7f6f241053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.128+0000 7f6f29800640 1 --2- 172.21.15.103:0/1645487729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24104f60 0x7f6f241053e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009860 tx=0x7f6f1002a340 comp rx=0 tx=0).stop 2024-09-16T01:57:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.128+0000 7f6f29800640 1 -- 172.21.15.103:0/1645487729 shutdown_connections 2024-09-16T01:57:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.128+0000 7f6f29800640 1 --2- 172.21.15.103:0/1645487729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24105920 0x7f6f2410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.128+0000 7f6f29800640 1 --2- 172.21.15.103:0/1645487729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24104f60 0x7f6f241053e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.128+0000 7f6f29800640 1 --2- 172.21.15.103:0/1645487729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24103e00 0x7f6f24104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.128+0000 7f6f29800640 1 -- 172.21.15.103:0/1645487729 >> 172.21.15.103:0/1645487729 conn(0x7f6f240ff5b0 msgr2=0x7f6f241019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.128+0000 7f6f29800640 1 -- 172.21.15.103:0/1645487729 shutdown_connections 2024-09-16T01:57:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.128+0000 7f6f29800640 1 -- 172.21.15.103:0/1645487729 wait complete. 2024-09-16T01:57:28.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.129+0000 7f6f29800640 1 Processor -- start 2024-09-16T01:57:28.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.129+0000 7f6f29800640 1 -- start start 2024-09-16T01:57:28.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.129+0000 7f6f29800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103e00 0x7f6f2419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:28.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.129+0000 7f6f29800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24104f60 0x7f6f2419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:28.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.129+0000 7f6f29800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24105920 0x7f6f241a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f29800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2410fab0 con 0x7f6f24103e00 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f29800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2410f930 con 0x7f6f24105920 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f29800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2410fc30 con 0x7f6f24104f60 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f22000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24104f60 0x7f6f2419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f22a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103e00 0x7f6f2419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f22000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24104f60 0x7f6f2419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:34648/0 (socket says 172.21.15.103:34648) 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f22000640 1 -- 172.21.15.103:0/2277500920 learned_addr learned my addr 172.21.15.103:0/2277500920 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f23400640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24105920 0x7f6f241a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f22000640 1 -- 172.21.15.103:0/2277500920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24105920 msgr2=0x7f6f241a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f22000640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24105920 0x7f6f241a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:28.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f22000640 1 -- 172.21.15.103:0/2277500920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103e00 msgr2=0x7f6f2419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:28.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f22000640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103e00 0x7f6f2419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:28.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f22000640 1 -- 172.21.15.103:0/2277500920 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f241a1d40 con 0x7f6f24104f60 2024-09-16T01:57:28.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.130+0000 7f6f23400640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24105920 0x7f6f241a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:57:28.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.131+0000 7f6f22000640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24104f60 0x7f6f2419a650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f10035040 tx=0x7f6f10033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:28.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.131+0000 7f6f1b400640 1 -- 172.21.15.103:0/2277500920 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10037040 con 0x7f6f24104f60 2024-09-16T01:57:28.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.131+0000 7f6f1b400640 1 -- 172.21.15.103:0/2277500920 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f10030850 con 0x7f6f24104f60 2024-09-16T01:57:28.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.131+0000 7f6f29800640 1 -- 172.21.15.103:0/2277500920 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f241a1fd0 con 0x7f6f24104f60 2024-09-16T01:57:28.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.131+0000 7f6f1b400640 1 -- 172.21.15.103:0/2277500920 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10004920 con 0x7f6f24104f60 2024-09-16T01:57:28.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.131+0000 7f6f29800640 1 -- 172.21.15.103:0/2277500920 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f241a2420 con 0x7f6f24104f60 2024-09-16T01:57:28.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.132+0000 7f6f1b400640 1 -- 172.21.15.103:0/2277500920 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f6f10004050 con 0x7f6f24104f60 2024-09-16T01:57:28.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.132+0000 7f6f29800640 1 -- 172.21.15.103:0/2277500920 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f2410b230 con 0x7f6f24104f60 2024-09-16T01:57:28.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.132+0000 7f6f1b400640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f08039500 0x7f6f0803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:28.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.133+0000 7f6f1b400640 1 -- 172.21.15.103:0/2277500920 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6f1006c5a0 con 0x7f6f24104f60 2024-09-16T01:57:28.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.133+0000 7f6f22a00640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f08039500 0x7f6f0803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:28.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.133+0000 7f6f22a00640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f08039500 0x7f6f0803b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009bd0 tx=0x7f6f14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:28.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.138+0000 7f6f1b400640 1 -- 172.21.15.103:0/2277500920 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f10004ac0 con 0x7f6f24104f60 2024-09-16T01:57:28.295 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:28.170+0000 7f0bf8f4b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:57:28.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:28.321+0000 7f6f29800640 1 -- 172.21.15.103:0/2277500920 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6f24062630 con 0x7f6f08039500 2024-09-16T01:57:28.623 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:28.310+0000 7f0bf8f4b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:57:28.623 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:28.620+0000 7f0bf8f4b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:57:29.045 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:28.680+0000 7f0bf8f4b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:57:29.045 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:28.864+0000 7f0bf8f4b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:57:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:57:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:57:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:57:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: Deploying daemon mgr.y on smithi136 2024-09-16T01:57:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:28 smithi172 ceph-mon[35340]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:29.049 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:29.049 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:29.049 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:29.049 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:57:29.049 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:57:29.049 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:57:29.049 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:29.049 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: Deploying daemon mgr.y on smithi136 2024-09-16T01:57:29.050 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:29.050 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:29.050 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:29.050 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:29.050 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:28 smithi103 ceph-mon[30398]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:29.095 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: Deploying daemon mgr.y on smithi136 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:29.097 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:28 smithi136 ceph-mon[35323]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:30.045 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:29 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:29.740+0000 7f0bf8f4b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:57:30.045 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:29 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:29.809+0000 7f0bf8f4b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:57:30.249 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:30.091+0000 7fe9e31f2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:57:30.249 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:30.247+0000 7fe9e31f2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:57:30.523 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:30.318+0000 7fe9e31f2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:57:30.523 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:30.472+0000 7fe9e31f2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:57:30.909 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:30 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:30.635+0000 7f0bf8f4b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:57:30.910 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:30 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:30.754+0000 7f0bf8f4b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:57:30.910 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:30 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:30.906+0000 7f0bf8f4b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:57:30.955 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:30.630+0000 7fe9e31f2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:57:30.956 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:30.953+0000 7fe9e31f2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:57:31.209 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.209 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.209 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.209 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.209 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:31.210 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-mon[35340]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:31.210 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:30 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:30.970+0000 7f0bf8f4b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:57:31.210 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:31.135+0000 7f0bf8f4b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:57:31.210 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:31.206+0000 7f0bf8f4b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:57:31.221 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:31 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:31.014+0000 7fe9e31f2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:57:31.221 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:31 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.221 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:31 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.222 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:31 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.222 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:31 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.222 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:31 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:31.222 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:31 smithi136 ceph-mon[35323]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:31.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:31 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:31 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:31 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:31 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:31.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:31 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:31.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:31 smithi103 ceph-mon[30398]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:31.524 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:31.275+0000 7f0bf8f4b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:57:31.524 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:31.441+0000 7f0bf8f4b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:57:31.524 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:31.508+0000 7f0bf8f4b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:57:31.525 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:31 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:31.218+0000 7fe9e31f2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:57:31.781 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:31.576+0000 7f0bf8f4b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:57:31.781 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:31.658+0000 7f0bf8f4b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:57:31.781 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:31.719+0000 7f0bf8f4b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:57:31.781 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:31.778+0000 7f0bf8f4b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2870746788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839e2a76-348d-4d7f-b43b-f446e829aade"}]: dispatch 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839e2a76-348d-4d7f-b43b-f446e829aade"}]: dispatch 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "839e2a76-348d-4d7f-b43b-f446e829aade"}]': finished 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T01:57:32.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:32.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:32.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:32 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3198325820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:57:32.523 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:32.110+0000 7fe9e31f2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:57:32.523 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:32.177+0000 7fe9e31f2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:57:32.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2870746788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839e2a76-348d-4d7f-b43b-f446e829aade"}]: dispatch 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839e2a76-348d-4d7f-b43b-f446e829aade"}]: dispatch 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "839e2a76-348d-4d7f-b43b-f446e829aade"}]': finished 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:32.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3198325820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:57:32.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:32.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:32.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:32.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2870746788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839e2a76-348d-4d7f-b43b-f446e829aade"}]: dispatch 2024-09-16T01:57:32.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "839e2a76-348d-4d7f-b43b-f446e829aade"}]: dispatch 2024-09-16T01:57:32.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "839e2a76-348d-4d7f-b43b-f446e829aade"}]': finished 2024-09-16T01:57:32.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T01:57:32.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:32.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:32.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3198325820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:57:33.045 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 01:57:32 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z[37142]: 2024-09-16T01:57:32.574+0000 7f0bf8f4b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:57:33.155 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:32.897+0000 7fe9e31f2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:57:33.155 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.011+0000 7fe9e31f2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:57:33.155 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.153+0000 7fe9e31f2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:57:33.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:33.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:33.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:33.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-mon[35323]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:33.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-mon[35323]: Standby manager daemon z started 2024-09-16T01:57:33.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-mon[35323]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:57:33.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-mon[35323]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:57:33.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-mon[35323]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:57:33.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-mon[35323]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:57:33.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:33.167+0000 7f6f1b400640 1 -- 172.21.15.103:0/2277500920 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7f6f10049ad0 con 0x7f6f24104f60 2024-09-16T01:57:33.432 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.215+0000 7fe9e31f2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:57:33.432 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.363+0000 7fe9e31f2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:57:33.433 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.430+0000 7fe9e31f2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:57:33.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:33 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:33.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:33 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:33.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:33 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:33.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:33 smithi103 ceph-mon[30398]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:33.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:33 smithi103 ceph-mon[30398]: Standby manager daemon z started 2024-09-16T01:57:33.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:33 smithi103 ceph-mon[30398]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:57:33.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:33 smithi103 ceph-mon[30398]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:57:33.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:33 smithi103 ceph-mon[30398]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:57:33.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:33 smithi103 ceph-mon[30398]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:57:33.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:33 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:33.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:33 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:33.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:33 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:33.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:33 smithi172 ceph-mon[35340]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:33.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:33 smithi172 ceph-mon[35340]: Standby manager daemon z started 2024-09-16T01:57:33.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:33 smithi172 ceph-mon[35340]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:57:33.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:33 smithi172 ceph-mon[35340]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:57:33.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:33 smithi172 ceph-mon[35340]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:57:33.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:33 smithi172 ceph-mon[35340]: from='mgr.? 172.21.15.172:0/437752157' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:57:33.715 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.490+0000 7fe9e31f2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:57:33.715 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.648+0000 7fe9e31f2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:57:33.715 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.713+0000 7fe9e31f2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:57:33.977 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.779+0000 7fe9e31f2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:57:33.978 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.860+0000 7fe9e31f2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:57:33.978 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.918+0000 7fe9e31f2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:57:33.978 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:33.975+0000 7fe9e31f2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:57:34.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:34 smithi136 ceph-mon[35323]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-16T01:57:34.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:34 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:57:34.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:34 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:34.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:34 smithi103 ceph-mon[30398]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-16T01:57:34.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:34 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:57:34.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:34 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:34.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:34 smithi172 ceph-mon[35340]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-16T01:57:34.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:34 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:57:34.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:34 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:35.023 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 01:57:34 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y[36686]: 2024-09-16T01:57:34.760+0000 7fe9e31f2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:57:35.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:35.178+0000 7f6f1b400640 1 -- 172.21.15.103:0/2277500920 <== mon.1 v2:172.21.15.172:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f6f10004fe0 con 0x7f6f24104f60 2024-09-16T01:57:35.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:35 smithi103 ceph-mon[30398]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:35.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:35 smithi103 ceph-mon[30398]: Standby manager daemon y started 2024-09-16T01:57:35.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:35 smithi103 ceph-mon[30398]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:57:35.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:35 smithi103 ceph-mon[30398]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:57:35.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:35 smithi103 ceph-mon[30398]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:57:35.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:35 smithi103 ceph-mon[30398]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:57:35.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:35 smithi136 ceph-mon[35323]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:35.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:35 smithi136 ceph-mon[35323]: Standby manager daemon y started 2024-09-16T01:57:35.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:35 smithi136 ceph-mon[35323]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:57:35.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:35 smithi136 ceph-mon[35323]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:57:35.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:35 smithi136 ceph-mon[35323]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:57:35.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:35 smithi136 ceph-mon[35323]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:57:35.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:35 smithi172 ceph-mon[35340]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:35.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:35 smithi172 ceph-mon[35340]: Standby manager daemon y started 2024-09-16T01:57:35.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:35 smithi172 ceph-mon[35340]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:57:35.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:35 smithi172 ceph-mon[35340]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:57:35.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:35 smithi172 ceph-mon[35340]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:57:35.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:35 smithi172 ceph-mon[35340]: from='mgr.? 172.21.15.136:0/1941354715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:57:36.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:36 smithi103 ceph-mon[30398]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-16T01:57:36.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:36 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:57:36.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:36 smithi136 ceph-mon[35323]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-16T01:57:36.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:36 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:57:36.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:36 smithi172 ceph-mon[35340]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-16T01:57:36.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:36 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:57:37.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:37 smithi103 ceph-mon[30398]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:37.446 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:37 smithi172 ceph-mon[35340]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:37.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:37 smithi136 ceph-mon[35323]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:39.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:39 smithi103 ceph-mon[30398]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:39.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:39 smithi136 ceph-mon[35323]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:39.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:39 smithi172 ceph-mon[35340]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:41 smithi103 ceph-mon[30398]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T01:57:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:41 smithi103 ceph-mon[30398]: Deploying daemon osd.0 on smithi103 2024-09-16T01:57:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:41 smithi136 ceph-mon[35323]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T01:57:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:41 smithi136 ceph-mon[35323]: Deploying daemon osd.0 on smithi103 2024-09-16T01:57:41.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:41 smithi172 ceph-mon[35340]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:41.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T01:57:41.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:41.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:41 smithi172 ceph-mon[35340]: Deploying daemon osd.0 on smithi103 2024-09-16T01:57:43.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:43 smithi103 ceph-mon[30398]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:43.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:43 smithi136 ceph-mon[35323]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:43.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:43 smithi172 ceph-mon[35340]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:45.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:45 smithi103 ceph-mon[30398]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:45.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:45 smithi136 ceph-mon[35323]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:45.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:45 smithi172 ceph-mon[35340]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:47.539 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:47 smithi103 ceph-mon[30398]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:47 smithi136 ceph-mon[35323]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:47 smithi172 ceph-mon[35340]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:48.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:48.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:48.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:48.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:48.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:48.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:48.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:48.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:48.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:49.700 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:49 smithi103 ceph-mon[30398]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:49 smithi136 ceph-mon[35323]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:49.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:49 smithi172 ceph-mon[35340]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:51.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:51.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:51.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:51.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:51.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:51.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:51 smithi136 ceph-mon[35323]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:51.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:51.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:51.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:51.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:51.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:51.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:51 smithi172 ceph-mon[35340]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:51 smithi103 ceph-mon[30398]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:53.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.055+0000 7f6f1b400640 1 -- 172.21.15.103:0/2277500920 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6f24062630 con 0x7f6f08039500 2024-09-16T01:57:53.057 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 0 on host 'smithi103' 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.057+0000 7f6f29800640 1 -- 172.21.15.103:0/2277500920 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f08039500 msgr2=0x7f6f0803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.057+0000 7f6f29800640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f08039500 0x7f6f0803b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009bd0 tx=0x7f6f14005f90 comp rx=0 tx=0).stop 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.057+0000 7f6f29800640 1 -- 172.21.15.103:0/2277500920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24104f60 msgr2=0x7f6f2419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.057+0000 7f6f29800640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24104f60 0x7f6f2419a650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f10035040 tx=0x7f6f10033d90 comp rx=0 tx=0).stop 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.057+0000 7f6f29800640 1 -- 172.21.15.103:0/2277500920 shutdown_connections 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.057+0000 7f6f29800640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f08039500 0x7f6f0803b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.057+0000 7f6f29800640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24105920 0x7f6f241a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.057+0000 7f6f29800640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24104f60 0x7f6f2419a650 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.057+0000 7f6f29800640 1 --2- 172.21.15.103:0/2277500920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103e00 0x7f6f2419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.057+0000 7f6f29800640 1 -- 172.21.15.103:0/2277500920 >> 172.21.15.103:0/2277500920 conn(0x7f6f240ff5b0 msgr2=0x7f6f24108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:53.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.058+0000 7f6f29800640 1 -- 172.21.15.103:0/2277500920 shutdown_connections 2024-09-16T01:57:53.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:57:53.058+0000 7f6f29800640 1 -- 172.21.15.103:0/2277500920 wait complete. 2024-09-16T01:57:53.747 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:53 smithi103 ceph-mon[30398]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:53.747 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:53.747 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:53.747 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:53 smithi136 ceph-mon[35323]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:53.795 DEBUG:teuthology.orchestra.run.smithi103:osd.0> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T01:57:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:53 smithi172 ceph-mon[35340]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:53.801 INFO:tasks.cephadm:Deploying osd.1 on smithi136 with /dev/vg_nvme/lv_4... 2024-09-16T01:57:53.801 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T01:57:54.052 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:57:54.979 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:57:54 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0[44237]: 2024-09-16T01:57:54.729+0000 7fb1f9833640 -1 osd.0 0 log_to_monitors true 2024-09-16T01:57:55.695 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:55 smithi136 ceph-mon[35323]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:55.695 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:55 smithi136 ceph-mon[35323]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T01:57:55.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:55 smithi103 ceph-mon[30398]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:55.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:55 smithi103 ceph-mon[30398]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T01:57:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:55 smithi172 ceph-mon[35340]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:55 smithi172 ceph-mon[35340]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T01:57:56.223 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:57:56.248 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_4 2024-09-16T01:57:56.499 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:57:56.713 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:56 smithi136 ceph-mon[35323]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T01:57:56.713 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:56 smithi136 ceph-mon[35323]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T01:57:56.713 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:56 smithi136 ceph-mon[35323]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T01:57:56.713 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:56.713 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:56 smithi136 ceph-mon[35323]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T01:57:56.713 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:56 smithi136 ceph-mon[35323]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T01:57:56.713 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:56.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:56 smithi103 ceph-mon[30398]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T01:57:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:56 smithi103 ceph-mon[30398]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T01:57:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:56 smithi103 ceph-mon[30398]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T01:57:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:56 smithi103 ceph-mon[30398]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T01:57:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:56 smithi103 ceph-mon[30398]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T01:57:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:56 smithi172 ceph-mon[35340]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T01:57:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:56 smithi172 ceph-mon[35340]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T01:57:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:56 smithi172 ceph-mon[35340]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T01:57:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:56 smithi172 ceph-mon[35340]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T01:57:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:56 smithi172 ceph-mon[35340]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T01:57:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:56.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.915+0000 7f5d11d08640 1 -- 172.21.15.136:0/1309625669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d0c102690 msgr2=0x7f5d0c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.915+0000 7f5d11d08640 1 --2- 172.21.15.136:0/1309625669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d0c102690 0x7f5d0c108f20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009ca0 tx=0x7f5d0402a8c0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.915+0000 7f5d11d08640 1 -- 172.21.15.136:0/1309625669 shutdown_connections 2024-09-16T01:57:56.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.915+0000 7f5d11d08640 1 --2- 172.21.15.136:0/1309625669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d0c102690 0x7f5d0c108f20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.915+0000 7f5d11d08640 1 --2- 172.21.15.136:0/1309625669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c101cd0 0x7f5d0c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.915+0000 7f5d11d08640 1 --2- 172.21.15.136:0/1309625669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c100ad0 0x7f5d0c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.915+0000 7f5d11d08640 1 -- 172.21.15.136:0/1309625669 >> 172.21.15.136:0/1309625669 conn(0x7f5d0c0fc280 msgr2=0x7f5d0c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.915+0000 7f5d11d08640 1 -- 172.21.15.136:0/1309625669 shutdown_connections 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.915+0000 7f5d11d08640 1 -- 172.21.15.136:0/1309625669 wait complete. 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.915+0000 7f5d11d08640 1 Processor -- start 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d11d08640 1 -- start start 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d11d08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c100ad0 0x7f5d0c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d11d08640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d0c101cd0 0x7f5d0c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d11d08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c102690 0x7f5d0c19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d11d08640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c10c750 con 0x7f5d0c100ad0 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d11d08640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c10c5d0 con 0x7f5d0c101cd0 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d11d08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c10c8d0 con 0x7f5d0c102690 2024-09-16T01:57:56.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d03fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d0c101cd0 0x7f5d0c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d03fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d0c101cd0 0x7f5d0c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:44360/0 (socket says 172.21.15.136:44360) 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d03fff640 1 -- 172.21.15.136:0/527316987 learned_addr learned my addr 172.21.15.136:0/527316987 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d0b7fe640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c100ad0 0x7f5d0c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d0bfff640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c102690 0x7f5d0c19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d03fff640 1 -- 172.21.15.136:0/527316987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c102690 msgr2=0x7f5d0c19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d03fff640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c102690 0x7f5d0c19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d03fff640 1 -- 172.21.15.136:0/527316987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c100ad0 msgr2=0x7f5d0c06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d03fff640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c100ad0 0x7f5d0c06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d03fff640 1 -- 172.21.15.136:0/527316987 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c19fb00 con 0x7f5d0c101cd0 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d0bfff640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c102690 0x7f5d0c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.916+0000 7f5d03fff640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d0c101cd0 0x7f5d0c19cbd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400d770 tx=0x7f5cf400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:56.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.917+0000 7f5d097fa640 1 -- 172.21.15.136:0/527316987 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4017070 con 0x7f5d0c101cd0 2024-09-16T01:57:56.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.917+0000 7f5d097fa640 1 -- 172.21.15.136:0/527316987 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf4005500 con 0x7f5d0c101cd0 2024-09-16T01:57:56.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.917+0000 7f5d11d08640 1 -- 172.21.15.136:0/527316987 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c19fd90 con 0x7f5d0c101cd0 2024-09-16T01:57:56.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.917+0000 7f5d097fa640 1 -- 172.21.15.136:0/527316987 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4004e00 con 0x7f5d0c101cd0 2024-09-16T01:57:56.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.917+0000 7f5d11d08640 1 -- 172.21.15.136:0/527316987 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c1a0220 con 0x7f5d0c101cd0 2024-09-16T01:57:56.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.918+0000 7f5d11d08640 1 -- 172.21.15.136:0/527316987 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0c107e80 con 0x7f5d0c101cd0 2024-09-16T01:57:56.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.920+0000 7f5d097fa640 1 -- 172.21.15.136:0/527316987 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f5cf4004fa0 con 0x7f5d0c101cd0 2024-09-16T01:57:56.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.920+0000 7f5d097fa640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5cf80a31b0 0x7f5cf80a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.921+0000 7f5d097fa640 1 -- 172.21.15.136:0/527316987 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f5cf40cdb90 con 0x7f5d0c101cd0 2024-09-16T01:57:56.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.921+0000 7f5d0b7fe640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5cf80a31b0 0x7f5cf80a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.921+0000 7f5d0b7fe640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5cf80a31b0 0x7f5cf80a5670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c101b30 tx=0x7f5cfc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:56.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:56.923+0000 7f5d097fa640 1 -- 172.21.15.136:0/527316987 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cf409c440 con 0x7f5d0c101cd0 2024-09-16T01:57:57.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:57:57.103+0000 7f5d11d08640 1 -- 172.21.15.136:0/527316987 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5d0c062630 con 0x7f5cf80a31b0 2024-09-16T01:57:57.453 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0[44237]: 2024-09-16T01:57:57.365+0000 7fb1f3600640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T01:57:57.453 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0[44237]: 2024-09-16T01:57:57.382+0000 7fb1ea000640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:57:57.751 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:57.751 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:57.751 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:57.751 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:57.751 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:57.751 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.751 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.751 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:57.752 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.752 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:57.752 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:57.752 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.752 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' 2024-09-16T01:57:57.752 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:57 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:57.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' 2024-09-16T01:57:57.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:57 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:57.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:57.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:57.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095]' entity='osd.0' 2024-09-16T01:57:57.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:57 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:58.707 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:58 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T01:57:58.707 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:58 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T01:57:58.707 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:58 smithi136 ceph-mon[35323]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:58.707 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:58 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi103 2024-09-16T01:57:58.707 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:58 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi103 to 17089M 2024-09-16T01:57:58.708 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:58 smithi136 ceph-mon[35323]: osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] boot 2024-09-16T01:57:58.708 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:58 smithi136 ceph-mon[35323]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T01:57:58.708 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:58.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:58 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T01:57:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:58 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T01:57:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:58 smithi103 ceph-mon[30398]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:58 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi103 2024-09-16T01:57:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:58 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi103 to 17089M 2024-09-16T01:57:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:58 smithi103 ceph-mon[30398]: osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] boot 2024-09-16T01:57:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:58 smithi103 ceph-mon[30398]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T01:57:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:58 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T01:57:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:58 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T01:57:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:58 smithi172 ceph-mon[35340]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:57:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:58 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi103 2024-09-16T01:57:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:58 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi103 to 17089M 2024-09-16T01:57:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:58 smithi172 ceph-mon[35340]: osd.0 [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] boot 2024-09-16T01:57:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:58 smithi172 ceph-mon[35340]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T01:57:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:57:59.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:59 smithi103 ceph-mon[30398]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:59 smithi103 ceph-mon[30398]: from='client.? 172.21.15.136:0/1806088708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393999bb-cc86-41ed-b0fa-7ae957c11374"}]: dispatch 2024-09-16T01:57:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:59 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393999bb-cc86-41ed-b0fa-7ae957c11374"}]: dispatch 2024-09-16T01:57:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:59 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "393999bb-cc86-41ed-b0fa-7ae957c11374"}]': finished 2024-09-16T01:57:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:59 smithi103 ceph-mon[30398]: osdmap e9: 2 total, 1 up, 2 in 2024-09-16T01:57:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:57:59 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:57:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:59 smithi136 ceph-mon[35323]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:59 smithi136 ceph-mon[35323]: from='client.? 172.21.15.136:0/1806088708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393999bb-cc86-41ed-b0fa-7ae957c11374"}]: dispatch 2024-09-16T01:57:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:59 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393999bb-cc86-41ed-b0fa-7ae957c11374"}]: dispatch 2024-09-16T01:57:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:59 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "393999bb-cc86-41ed-b0fa-7ae957c11374"}]': finished 2024-09-16T01:57:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:59 smithi136 ceph-mon[35323]: osdmap e9: 2 total, 1 up, 2 in 2024-09-16T01:57:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:57:59 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:59 smithi172 ceph-mon[35340]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:59 smithi172 ceph-mon[35340]: from='client.? 172.21.15.136:0/1806088708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393999bb-cc86-41ed-b0fa-7ae957c11374"}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:59 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393999bb-cc86-41ed-b0fa-7ae957c11374"}]: dispatch 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:59 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "393999bb-cc86-41ed-b0fa-7ae957c11374"}]': finished 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:59 smithi172 ceph-mon[35340]: osdmap e9: 2 total, 1 up, 2 in 2024-09-16T01:57:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:57:59 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:00.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:00 smithi103 ceph-mon[30398]: from='client.? 172.21.15.136:0/1448252742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:00.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:00 smithi136 ceph-mon[35323]: from='client.? 172.21.15.136:0/1448252742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:00.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:00 smithi172 ceph-mon[35340]: from='client.? 172.21.15.136:0/1448252742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:01.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:01 smithi103 ceph-mon[30398]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:01.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:01 smithi136 ceph-mon[35323]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:01.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:01 smithi172 ceph-mon[35340]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:03.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:03 smithi103 ceph-mon[30398]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:03.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:03 smithi136 ceph-mon[35323]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:03.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:03 smithi172 ceph-mon[35340]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:05.733 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:05 smithi136 ceph-mon[35323]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:05.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:05 smithi103 ceph-mon[30398]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:05.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:05 smithi172 ceph-mon[35340]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:07.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:07 smithi136 ceph-mon[35323]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:07.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T01:58:07.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:07.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:07 smithi172 ceph-mon[35340]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:07.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T01:58:07.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:08.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:07 smithi103 ceph-mon[30398]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:08.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T01:58:08.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:08.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:08 smithi136 ceph-mon[35323]: Deploying daemon osd.1 on smithi136 2024-09-16T01:58:08.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:08 smithi172 ceph-mon[35340]: Deploying daemon osd.1 on smithi136 2024-09-16T01:58:09.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:08 smithi103 ceph-mon[30398]: Deploying daemon osd.1 on smithi136 2024-09-16T01:58:09.763 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:09 smithi136 ceph-mon[35323]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:09.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:09 smithi172 ceph-mon[35340]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:10.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:09 smithi103 ceph-mon[30398]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:11 smithi136 ceph-mon[35323]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:11 smithi172 ceph-mon[35340]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:12.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:11 smithi103 ceph-mon[30398]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:13.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:13 smithi136 ceph-mon[35323]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:13 smithi172 ceph-mon[35340]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:14.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:13 smithi103 ceph-mon[30398]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:14.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:14 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:14.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:14 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:14.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:14 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:14.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:14 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:14.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:14 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:14.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:14 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:15.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:14 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:15.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:14 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:15.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:14 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:15.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:15 smithi136 ceph-mon[35323]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:15.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:15 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:15.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:15 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:15.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:15 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:15.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:15 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:15.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:15 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:16.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:15 smithi103 ceph-mon[30398]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:16.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:15 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:16.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:15 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:16.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:15 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:16.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:15 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:16.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:15 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:15 smithi172 ceph-mon[35340]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:15 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:15 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:16.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:15 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:16.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:15 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:16.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:15 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:17.557 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.554+0000 7f5d097fa640 1 -- 172.21.15.136:0/527316987 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5d0c062630 con 0x7f5cf80a31b0 2024-09-16T01:58:17.557 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 1 on host 'smithi136' 2024-09-16T01:58:17.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.557+0000 7f5d11d08640 1 -- 172.21.15.136:0/527316987 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5cf80a31b0 msgr2=0x7f5cf80a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:17.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.557+0000 7f5d11d08640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5cf80a31b0 0x7f5cf80a5670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c101b30 tx=0x7f5cfc00b390 comp rx=0 tx=0).stop 2024-09-16T01:58:17.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.557+0000 7f5d11d08640 1 -- 172.21.15.136:0/527316987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d0c101cd0 msgr2=0x7f5d0c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:17.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.557+0000 7f5d11d08640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d0c101cd0 0x7f5d0c19cbd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400d770 tx=0x7f5cf400dc30 comp rx=0 tx=0).stop 2024-09-16T01:58:17.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.557+0000 7f5d11d08640 1 -- 172.21.15.136:0/527316987 shutdown_connections 2024-09-16T01:58:17.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.557+0000 7f5d11d08640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5cf80a31b0 0x7f5cf80a5670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:17.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.557+0000 7f5d11d08640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c102690 0x7f5d0c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:17.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.557+0000 7f5d11d08640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d0c101cd0 0x7f5d0c19cbd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:17.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.557+0000 7f5d11d08640 1 --2- 172.21.15.136:0/527316987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c100ad0 0x7f5d0c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:17.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.557+0000 7f5d11d08640 1 -- 172.21.15.136:0/527316987 >> 172.21.15.136:0/527316987 conn(0x7f5d0c0fc280 msgr2=0x7f5d0c105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:17.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.558+0000 7f5d11d08640 1 -- 172.21.15.136:0/527316987 shutdown_connections 2024-09-16T01:58:17.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:58:17.558+0000 7f5d11d08640 1 -- 172.21.15.136:0/527316987 wait complete. 2024-09-16T01:58:17.807 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:17 smithi103 ceph-mon[30398]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:17.808 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:17.808 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:17.835 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:17 smithi136 ceph-mon[35323]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:17.835 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:17.835 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:18.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:17 smithi172 ceph-mon[35340]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:18.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:18.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:18.144 DEBUG:teuthology.orchestra.run.smithi136:osd.1> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T01:58:18.147 INFO:tasks.cephadm:Deploying osd.2 on smithi172 with /dev/vg_nvme/lv_4... 2024-09-16T01:58:18.147 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T01:58:18.399 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:58:18.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:18.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:18 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:18.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:18 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:19.680 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:19 smithi172 ceph-mon[35340]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:19.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:19 smithi103 ceph-mon[30398]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:19 smithi136 ceph-mon[35323]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:20.189 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:19 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1[39801]: 2024-09-16T01:58:19.936+0000 7f78d927d640 -1 osd.1 0 log_to_monitors true 2024-09-16T01:58:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:20 smithi136 ceph-mon[35323]: from='osd.1 [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:58:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:20 smithi136 ceph-mon[35323]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:58:20.897 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:20 smithi172 ceph-mon[35340]: from='osd.1 [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:58:20.897 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:20 smithi172 ceph-mon[35340]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:58:20.901 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:58:20.934 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_4 2024-09-16T01:58:21.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:20 smithi103 ceph-mon[30398]: from='osd.1 [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:58:21.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:20 smithi103 ceph-mon[30398]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:58:21.186 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:58:21.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.642+0000 7efd5aeb0640 1 -- 172.21.15.172:0/4123333099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd54102eb0 msgr2=0x7efd54103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:21.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.642+0000 7efd5aeb0640 1 --2- 172.21.15.172:0/4123333099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd54102eb0 0x7efd54103330 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd40009930 tx=0x7efd4002a420 comp rx=0 tx=0).stop 2024-09-16T01:58:21.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.642+0000 7efd5aeb0640 1 -- 172.21.15.172:0/4123333099 shutdown_connections 2024-09-16T01:58:21.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.642+0000 7efd5aeb0640 1 --2- 172.21.15.172:0/4123333099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54103870 0x7efd5410a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:21.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.642+0000 7efd5aeb0640 1 --2- 172.21.15.172:0/4123333099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd54102eb0 0x7efd54103330 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:21.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.642+0000 7efd5aeb0640 1 --2- 172.21.15.172:0/4123333099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd540ffed0 0x7efd541002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:21.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.642+0000 7efd5aeb0640 1 -- 172.21.15.172:0/4123333099 >> 172.21.15.172:0/4123333099 conn(0x7efd540fba40 msgr2=0x7efd540fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:21.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.642+0000 7efd5aeb0640 1 -- 172.21.15.172:0/4123333099 shutdown_connections 2024-09-16T01:58:21.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.642+0000 7efd5aeb0640 1 -- 172.21.15.172:0/4123333099 wait complete. 2024-09-16T01:58:21.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5aeb0640 1 Processor -- start 2024-09-16T01:58:21.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5aeb0640 1 -- start start 2024-09-16T01:58:21.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5aeb0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd540ffed0 0x7efd54198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:21.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5aeb0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54102eb0 0x7efd54198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5aeb0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd54103870 0x7efd5419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5aeb0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efd5410d9a0 con 0x7efd540ffed0 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5aeb0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd5410d820 con 0x7efd54102eb0 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5aeb0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd5410db20 con 0x7efd54103870 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5944d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd54103870 0x7efd5419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5944d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd54103870 0x7efd5419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46952/0 (socket says 172.21.15.172:46952) 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd53fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54102eb0 0x7efd54198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd5944d640 1 -- 172.21.15.172:0/3645685680 learned_addr learned my addr 172.21.15.172:0/3645685680 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.643+0000 7efd58c4c640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd540ffed0 0x7efd54198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd53fff640 1 -- 172.21.15.172:0/3645685680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd54103870 msgr2=0x7efd5419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:21.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd53fff640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd54103870 0x7efd5419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd53fff640 1 -- 172.21.15.172:0/3645685680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd540ffed0 msgr2=0x7efd54198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd53fff640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd540ffed0 0x7efd54198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd53fff640 1 -- 172.21.15.172:0/3645685680 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5419fc60 con 0x7efd54102eb0 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd5944d640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd54103870 0x7efd5419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd53fff640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54102eb0 0x7efd54198570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efd4002a940 tx=0x7efd40032c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd51ffb640 1 -- 172.21.15.172:0/3645685680 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4003f030 con 0x7efd54102eb0 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd51ffb640 1 -- 172.21.15.172:0/3645685680 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd40032e50 con 0x7efd54102eb0 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd5aeb0640 1 -- 172.21.15.172:0/3645685680 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5419fef0 con 0x7efd54102eb0 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd51ffb640 1 -- 172.21.15.172:0/3645685680 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40035070 con 0x7efd54102eb0 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.644+0000 7efd5aeb0640 1 -- 172.21.15.172:0/3645685680 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd541a03c0 con 0x7efd54102eb0 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.645+0000 7efd5aeb0640 1 -- 172.21.15.172:0/3645685680 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd54109100 con 0x7efd54102eb0 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.646+0000 7efd51ffb640 1 -- 172.21.15.172:0/3645685680 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7efd40042030 con 0x7efd54102eb0 2024-09-16T01:58:21.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.647+0000 7efd51ffb640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd440a31e0 0x7efd440a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:21.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.647+0000 7efd51ffb640 1 -- 172.21.15.172:0/3645685680 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2029+0+0 (secure 0 0 0) 0x7efd400ee370 con 0x7efd54102eb0 2024-09-16T01:58:21.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.647+0000 7efd58c4c640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd440a31e0 0x7efd440a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:21.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.648+0000 7efd58c4c640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd440a31e0 0x7efd440a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd5406a590 tx=0x7efd48009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:21.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.651+0000 7efd51ffb640 1 -- 172.21.15.172:0/3645685680 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd400bc960 con 0x7efd54102eb0 2024-09-16T01:58:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T01:58:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T01:58:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='osd.1 [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:58:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:58:21.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:21.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:21.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:21.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:21.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:21.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:21.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:21 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:21.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:21.826+0000 7efd5aeb0640 1 -- 172.21.15.172:0/3645685680 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7efd54062630 con 0x7efd440a31e0 2024-09-16T01:58:21.827 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:21.827 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T01:58:21.827 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T01:58:21.828 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:21.828 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='osd.1 [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:58:21.828 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:58:21.828 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:21.828 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:21.828 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:21.828 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:21.828 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:21.828 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:21.828 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:21 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T01:58:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T01:58:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='osd.1 [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:58:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:58:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:22.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:22.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:22.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:21 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:22.523 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1[39801]: 2024-09-16T01:58:22.239+0000 7f78d49dd640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T01:58:22.523 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1[39801]: 2024-09-16T01:58:22.246+0000 7f78cd00f640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:58:22.825 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:22 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi136 2024-09-16T01:58:22.825 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:22 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi136 to 17088M 2024-09-16T01:58:22.825 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:22 smithi172 ceph-mon[35340]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T01:58:22.825 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:22 smithi172 ceph-mon[35340]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T01:58:22.825 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:22.825 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:22.825 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:22.825 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:22.825 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:22 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:22.826 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:22 smithi172 ceph-mon[35340]: from='osd.1 ' entity='osd.1' 2024-09-16T01:58:23.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:22 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi136 2024-09-16T01:58:23.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:22 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi136 to 17088M 2024-09-16T01:58:23.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:22 smithi103 ceph-mon[30398]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T01:58:23.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:22 smithi103 ceph-mon[30398]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T01:58:23.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:23.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:23.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:23.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:23.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:22 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:23.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:22 smithi103 ceph-mon[30398]: from='osd.1 ' entity='osd.1' 2024-09-16T01:58:23.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi136 2024-09-16T01:58:23.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi136 to 17088M 2024-09-16T01:58:23.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-mon[35323]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T01:58:23.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-mon[35323]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T01:58:23.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:23.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:23.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:23.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:23.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:23.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:22 smithi136 ceph-mon[35323]: from='osd.1 ' entity='osd.1' 2024-09-16T01:58:23.773 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:23 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T01:58:23.774 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:23 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T01:58:23.774 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:23 smithi172 ceph-mon[35340]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:23.774 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:23 smithi172 ceph-mon[35340]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:23.774 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:23.774 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:23 smithi172 ceph-mon[35340]: osd.1 [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] boot 2024-09-16T01:58:23.774 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:23 smithi172 ceph-mon[35340]: osdmap e12: 2 total, 2 up, 2 in 2024-09-16T01:58:23.774 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:23 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T01:58:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:23 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T01:58:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:23 smithi103 ceph-mon[30398]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:23 smithi103 ceph-mon[30398]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:23 smithi103 ceph-mon[30398]: osd.1 [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] boot 2024-09-16T01:58:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:23 smithi103 ceph-mon[30398]: osdmap e12: 2 total, 2 up, 2 in 2024-09-16T01:58:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:23 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T01:58:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:23 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T01:58:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:23 smithi136 ceph-mon[35323]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:23 smithi136 ceph-mon[35323]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:58:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:23 smithi136 ceph-mon[35323]: osd.1 [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] boot 2024-09-16T01:58:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:23 smithi136 ceph-mon[35323]: osdmap e12: 2 total, 2 up, 2 in 2024-09-16T01:58:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:58:25.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:24 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/137432381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a337ce9d-76c5-4410-b843-fce557c1401e"}]: dispatch 2024-09-16T01:58:25.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:24 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a337ce9d-76c5-4410-b843-fce557c1401e"}]: dispatch 2024-09-16T01:58:25.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:24 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a337ce9d-76c5-4410-b843-fce557c1401e"}]': finished 2024-09-16T01:58:25.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:24 smithi103 ceph-mon[30398]: osdmap e13: 3 total, 2 up, 3 in 2024-09-16T01:58:25.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:24 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/137432381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a337ce9d-76c5-4410-b843-fce557c1401e"}]: dispatch 2024-09-16T01:58:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:24 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a337ce9d-76c5-4410-b843-fce557c1401e"}]: dispatch 2024-09-16T01:58:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:24 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a337ce9d-76c5-4410-b843-fce557c1401e"}]': finished 2024-09-16T01:58:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:24 smithi136 ceph-mon[35323]: osdmap e13: 3 total, 2 up, 3 in 2024-09-16T01:58:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:24 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/137432381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a337ce9d-76c5-4410-b843-fce557c1401e"}]: dispatch 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:24 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a337ce9d-76c5-4410-b843-fce557c1401e"}]: dispatch 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:24 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a337ce9d-76c5-4410-b843-fce557c1401e"}]': finished 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:24 smithi172 ceph-mon[35340]: osdmap e13: 3 total, 2 up, 3 in 2024-09-16T01:58:25.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:26.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:25 smithi103 ceph-mon[30398]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:26.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:25 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/3068101538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:25 smithi136 ceph-mon[35323]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:25 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/3068101538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:25 smithi172 ceph-mon[35340]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:25 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/3068101538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:28.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:27 smithi103 ceph-mon[30398]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:27 smithi136 ceph-mon[35323]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:27 smithi172 ceph-mon[35340]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:29.758 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:29 smithi172 ceph-mon[35340]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:30.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:29 smithi103 ceph-mon[30398]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:29 smithi136 ceph-mon[35323]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:31.757 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:31 smithi172 ceph-mon[35340]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:32.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:31 smithi103 ceph-mon[30398]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:31 smithi136 ceph-mon[35323]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:33.784 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:33 smithi172 ceph-mon[35340]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:33.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:33 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T01:58:33.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:33 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:34.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:33 smithi103 ceph-mon[30398]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:34.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:33 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T01:58:34.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:33 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:33 smithi136 ceph-mon[35323]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:33 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T01:58:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:33 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:34.655 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:34 smithi172 ceph-mon[35340]: Deploying daemon osd.2 on smithi172 2024-09-16T01:58:35.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:34 smithi103 ceph-mon[30398]: Deploying daemon osd.2 on smithi172 2024-09-16T01:58:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:34 smithi136 ceph-mon[35323]: Deploying daemon osd.2 on smithi172 2024-09-16T01:58:36.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:35 smithi103 ceph-mon[30398]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:36.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:35 smithi136 ceph-mon[35323]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:36.043 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:35 smithi172 ceph-mon[35340]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:37.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:36 smithi103 ceph-mon[30398]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:37.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:36 smithi136 ceph-mon[35323]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:37.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:36 smithi172 ceph-mon[35340]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:39.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:39 smithi172 ceph-mon[35340]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:39.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:39 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:39.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:39 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:39.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:39 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:39.726 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:39 smithi136 ceph-mon[35323]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:39.726 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:39 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:39.726 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:39 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:39.726 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:39 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:39.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:39 smithi103 ceph-mon[30398]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:39.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:39 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:39.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:39 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:39.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:39 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:41.506 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:41 smithi172 ceph-mon[35340]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:41.507 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:41.507 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:41.507 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:41.507 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:41.507 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:41.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:41 smithi103 ceph-mon[30398]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:41 smithi136 ceph-mon[35323]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:43.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.341+0000 7efd51ffb640 1 -- 172.21.15.172:0/3645685680 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efd54062630 con 0x7efd440a31e0 2024-09-16T01:58:43.343 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 2 on host 'smithi172' 2024-09-16T01:58:43.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 -- 172.21.15.172:0/3645685680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd440a31e0 msgr2=0x7efd440a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd440a31e0 0x7efd440a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd5406a590 tx=0x7efd48009110 comp rx=0 tx=0).stop 2024-09-16T01:58:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 -- 172.21.15.172:0/3645685680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54102eb0 msgr2=0x7efd54198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54102eb0 0x7efd54198570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efd4002a940 tx=0x7efd40032c60 comp rx=0 tx=0).stop 2024-09-16T01:58:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 -- 172.21.15.172:0/3645685680 shutdown_connections 2024-09-16T01:58:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd440a31e0 0x7efd440a56a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd54103870 0x7efd5419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54102eb0 0x7efd54198570 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 --2- 172.21.15.172:0/3645685680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd540ffed0 0x7efd54198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 -- 172.21.15.172:0/3645685680 >> 172.21.15.172:0/3645685680 conn(0x7efd540fba40 msgr2=0x7efd541062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 -- 172.21.15.172:0/3645685680 shutdown_connections 2024-09-16T01:58:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:58:43.344+0000 7efd5aeb0640 1 -- 172.21.15.172:0/3645685680 wait complete. 2024-09-16T01:58:43.679 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:43 smithi172 ceph-mon[35340]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:43.680 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:43 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:43.680 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:43 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:43.680 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:43 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:43.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:43 smithi103 ceph-mon[30398]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:43.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:43 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:43.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:43 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:43.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:43 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:43.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:43 smithi136 ceph-mon[35323]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:43.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:43 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:43.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:43 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:43.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:43 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:43.999 DEBUG:teuthology.orchestra.run.smithi172:osd.2> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T01:58:44.002 INFO:tasks.cephadm:Deploying osd.3 on smithi103 with /dev/vg_nvme/lv_3... 2024-09-16T01:58:44.003 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T01:58:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:45.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:45 smithi103 ceph-mon[30398]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:45.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:45 smithi136 ceph-mon[35323]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:45.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:45 smithi172 ceph-mon[35340]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:45.795 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2[40339]: 2024-09-16T01:58:45.473+0000 7f6fd4715640 -1 osd.2 0 log_to_monitors true 2024-09-16T01:58:46.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:46 smithi103 ceph-mon[30398]: from='osd.2 [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:58:46.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:46 smithi103 ceph-mon[30398]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:58:46.759 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:46 smithi172 ceph-mon[35340]: from='osd.2 [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:58:46.759 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:46 smithi172 ceph-mon[35340]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:58:46.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:46 smithi136 ceph-mon[35323]: from='osd.2 [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:58:46.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:46 smithi136 ceph-mon[35323]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:58:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T01:58:47.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-09-16T01:58:47.712 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:47.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='osd.2 [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:47 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T01:58:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T01:58:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='osd.2 [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T01:58:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T01:58:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:47.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:47.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:47.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:47 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T01:58:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T01:58:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='osd.2 [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T01:58:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T01:58:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:58:47.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:47.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:47.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:47.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:47 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:48.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.272+0000 7f5b0d800640 1 -- 172.21.15.103:0/2593562201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b08109a80 msgr2=0x7f5b0810bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:48.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.272+0000 7f5b0d800640 1 --2- 172.21.15.103:0/2593562201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b08109a80 0x7f5b0810bf40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009ca0 tx=0x7f5b0002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.272+0000 7f5b0d800640 1 -- 172.21.15.103:0/2593562201 shutdown_connections 2024-09-16T01:58:48.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.272+0000 7f5b0d800640 1 --2- 172.21.15.103:0/2593562201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b08109a80 0x7f5b0810bf40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.272+0000 7f5b0d800640 1 --2- 172.21.15.103:0/2593562201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b08074c50 0x7f5b080750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.272+0000 7f5b0d800640 1 --2- 172.21.15.103:0/2593562201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b08076220 0x7f5b08074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.272+0000 7f5b0d800640 1 -- 172.21.15.103:0/2593562201 >> 172.21.15.103:0/2593562201 conn(0x7f5b080fd390 msgr2=0x7f5b080ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:48.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.272+0000 7f5b0d800640 1 -- 172.21.15.103:0/2593562201 shutdown_connections 2024-09-16T01:58:48.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.272+0000 7f5b0d800640 1 -- 172.21.15.103:0/2593562201 wait complete. 2024-09-16T01:58:48.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.273+0000 7f5b0d800640 1 Processor -- start 2024-09-16T01:58:48.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.273+0000 7f5b0d800640 1 -- start start 2024-09-16T01:58:48.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.273+0000 7f5b0d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b08074c50 0x7f5b0819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:48.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.273+0000 7f5b0d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b08076220 0x7f5b0819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:48.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.273+0000 7f5b0d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b08109a80 0x7f5b081a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:48.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.273+0000 7f5b0d800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0810faf0 con 0x7f5b08074c50 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.273+0000 7f5b0d800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0810f970 con 0x7f5b08076220 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.273+0000 7f5b0d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0810fc70 con 0x7f5b08109a80 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b07400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b08109a80 0x7f5b081a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b06a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b08074c50 0x7f5b0819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b06000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b08076220 0x7f5b0819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b07400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b08109a80 0x7f5b081a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:50298/0 (socket says 172.21.15.103:50298) 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b07400640 1 -- 172.21.15.103:0/1756342432 learned_addr learned my addr 172.21.15.103:0/1756342432 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b06a00640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b08074c50 0x7f5b0819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46710/0 (socket says 172.21.15.103:46710) 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b06a00640 1 -- 172.21.15.103:0/1756342432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b08109a80 msgr2=0x7f5b081a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b06a00640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b08109a80 0x7f5b081a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b06a00640 1 -- 172.21.15.103:0/1756342432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b08076220 msgr2=0x7f5b0819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:48.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b06a00640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b08076220 0x7f5b0819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b06a00640 1 -- 172.21.15.103:0/1756342432 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b081a1cf0 con 0x7f5b08074c50 2024-09-16T01:58:48.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.274+0000 7f5b06a00640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b08074c50 0x7f5b0819a0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5af800e9c0 tx=0x7f5af800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:48.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.275+0000 7f5aff400640 1 -- 172.21.15.103:0/1756342432 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af800ccc0 con 0x7f5b08074c50 2024-09-16T01:58:48.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.275+0000 7f5aff400640 1 -- 172.21.15.103:0/1756342432 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af8010910 con 0x7f5b08074c50 2024-09-16T01:58:48.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.275+0000 7f5b0d800640 1 -- 172.21.15.103:0/1756342432 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b081a1fe0 con 0x7f5b08074c50 2024-09-16T01:58:48.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.275+0000 7f5aff400640 1 -- 172.21.15.103:0/1756342432 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af80188c0 con 0x7f5b08074c50 2024-09-16T01:58:48.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.275+0000 7f5b0d800640 1 -- 172.21.15.103:0/1756342432 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b081a2430 con 0x7f5b08074c50 2024-09-16T01:58:48.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.276+0000 7f5b0d800640 1 -- 172.21.15.103:0/1756342432 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b08108800 con 0x7f5b08074c50 2024-09-16T01:58:48.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.278+0000 7f5aff400640 1 -- 172.21.15.103:0/1756342432 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f5af8010ac0 con 0x7f5b08074c50 2024-09-16T01:58:48.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.279+0000 7f5aff400640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5aec0a31e0 0x7f5aec0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:48.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.279+0000 7f5aff400640 1 -- 172.21.15.103:0/1756342432 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2571+0+0 (secure 0 0 0) 0x7f5af80ce500 con 0x7f5b08074c50 2024-09-16T01:58:48.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.279+0000 7f5b06000640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5aec0a31e0 0x7f5aec0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:48.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.280+0000 7f5b06000640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5aec0a31e0 0x7f5aec0a56a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b0819b5e0 tx=0x7f5af4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.284+0000 7f5aff400640 1 -- 172.21.15.103:0/1756342432 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5af809c280 con 0x7f5b08074c50 2024-09-16T01:58:48.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:48.469+0000 7f5b0d800640 1 -- 172.21.15.103:0/1756342432 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5b08062630 con 0x7f5aec0a31e0 2024-09-16T01:58:48.545 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2[40339]: 2024-09-16T01:58:48.265+0000 7f6fcfe75640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T01:58:48.545 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2[40339]: 2024-09-16T01:58:48.272+0000 7f6fc84a7640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:58:48.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:48 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi172 2024-09-16T01:58:48.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:48 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-16T01:58:48.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:48 smithi172 ceph-mon[35340]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T01:58:48.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:48 smithi172 ceph-mon[35340]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T01:58:48.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:48.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:48.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:48 smithi172 ceph-mon[35340]: from='osd.2 ' entity='osd.2' 2024-09-16T01:58:48.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:48 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi172 2024-09-16T01:58:48.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:48 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-16T01:58:48.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:48 smithi103 ceph-mon[30398]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T01:58:48.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:48 smithi103 ceph-mon[30398]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T01:58:48.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:48.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:48.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:48 smithi103 ceph-mon[30398]: from='osd.2 ' entity='osd.2' 2024-09-16T01:58:48.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:48 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi172 2024-09-16T01:58:48.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:48 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-16T01:58:48.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:48 smithi136 ceph-mon[35323]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T01:58:48.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:48 smithi136 ceph-mon[35323]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T01:58:48.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:48.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:48.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:48 smithi136 ceph-mon[35323]: from='osd.2 ' entity='osd.2' 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: osd.2 [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] boot 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: osdmap e16: 3 total, 3 up, 3 in 2024-09-16T01:58:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T01:58:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T01:58:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: osd.2 [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] boot 2024-09-16T01:58:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: osdmap e16: 3 total, 3 up, 3 in 2024-09-16T01:58:49.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:49.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T01:58:49.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T01:58:49.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:58:49.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:49.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:49.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:58:49.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:58:49.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:49.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: osd.2 [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] boot 2024-09-16T01:58:49.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: osdmap e16: 3 total, 3 up, 3 in 2024-09-16T01:58:49.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:58:51.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:51 smithi136 ceph-mon[35323]: pgmap v77: 0 pgs: ; 0 B data, 492 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:51.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:58:51.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T01:58:51.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:51 smithi136 ceph-mon[35323]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T01:58:51.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:58:51.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:51 smithi172 ceph-mon[35340]: pgmap v77: 0 pgs: ; 0 B data, 492 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:51.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:58:51.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T01:58:51.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:51 smithi172 ceph-mon[35340]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T01:58:51.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:58:51.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:51 smithi103 ceph-mon[30398]: pgmap v77: 0 pgs: ; 0 B data, 492 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:51.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:58:51.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T01:58:51.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:51 smithi103 ceph-mon[30398]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T01:58:51.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:58:52.141 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:52 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1423528690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2735e5b-dd14-483d-84de-a0172d862b73"}]: dispatch 2024-09-16T01:58:52.141 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:52 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:58:52.141 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:52 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1423528690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2735e5b-dd14-483d-84de-a0172d862b73"}]': finished 2024-09-16T01:58:52.141 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:52 smithi103 ceph-mon[30398]: osdmap e18: 4 total, 3 up, 4 in 2024-09-16T01:58:52.141 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:52 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:58:52.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:52 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1423528690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2735e5b-dd14-483d-84de-a0172d862b73"}]: dispatch 2024-09-16T01:58:52.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:52 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:58:52.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:52 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1423528690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2735e5b-dd14-483d-84de-a0172d862b73"}]': finished 2024-09-16T01:58:52.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:52 smithi136 ceph-mon[35323]: osdmap e18: 4 total, 3 up, 4 in 2024-09-16T01:58:52.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:52 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:58:52.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:52 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1423528690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2735e5b-dd14-483d-84de-a0172d862b73"}]: dispatch 2024-09-16T01:58:52.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:52 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:58:52.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:52 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1423528690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2735e5b-dd14-483d-84de-a0172d862b73"}]': finished 2024-09-16T01:58:52.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:52 smithi172 ceph-mon[35340]: osdmap e18: 4 total, 3 up, 4 in 2024-09-16T01:58:52.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:52 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:58:52.504 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:58:52 smithi103 sudo[49153]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:58:52.505 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:58:52 smithi103 sudo[49153]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:52.505 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:58:52 smithi103 sudo[49153]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:52.505 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:58:52 smithi103 sudo[49153]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:53.005 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:58:52 smithi103 sudo[49182]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:58:53.005 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:58:52 smithi103 sudo[49182]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:53.005 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:58:52 smithi103 sudo[49182]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:53.005 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 01:58:52 smithi103 sudo[49182]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:53.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:58:53.022+0000 7f5aff400640 1 -- 172.21.15.103:0/1756342432 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5af80988f0 con 0x7f5b08074c50 2024-09-16T01:58:53.273 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:53 smithi136 sudo[43804]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:58:53.273 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:53 smithi136 sudo[43804]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:53.273 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:53 smithi136 sudo[43804]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:53.273 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:53 smithi136 sudo[43804]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:53.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:53 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/949343079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:53.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:53 smithi136 ceph-mon[35323]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 492 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:53.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:53 smithi136 ceph-mon[35323]: osdmap e19: 4 total, 3 up, 4 in 2024-09-16T01:58:53.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:58:53.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:53.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:53 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/949343079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:53.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:53 smithi172 ceph-mon[35340]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 492 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:53.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:53 smithi172 ceph-mon[35340]: osdmap e19: 4 total, 3 up, 4 in 2024-09-16T01:58:53.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:58:53.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:53.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:53 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/949343079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:58:53.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:53 smithi103 ceph-mon[30398]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 492 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:53.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:53 smithi103 ceph-mon[30398]: osdmap e19: 4 total, 3 up, 4 in 2024-09-16T01:58:53.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:58:53.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:53.773 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:53 smithi136 sudo[43807]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:58:53.773 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:53 smithi136 sudo[43807]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:53.773 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:53 smithi136 sudo[43807]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:53.773 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 01:58:53 smithi136 sudo[43807]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:53.976 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:53 smithi172 sudo[44429]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:58:53.976 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:53 smithi172 sudo[44429]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:53.976 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:53 smithi172 sudo[44429]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:53.976 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:53 smithi172 sudo[44429]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:54.295 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:53 smithi172 sudo[44432]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:58:54.295 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:53 smithi172 sudo[44432]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:54.295 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:53 smithi172 sudo[44432]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:54.295 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 01:58:53 smithi172 sudo[44432]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:55.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:54 smithi103 sudo[49594]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:58:55.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:54 smithi103 sudo[49594]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:55.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:54 smithi103 sudo[49594]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:55.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:54 smithi103 sudo[49594]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:55.340 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 ceph-mon[35340]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T01:58:55.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:55.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:55.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 ceph-mon[35340]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:58:55.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 sudo[44435]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:58:55.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 sudo[44435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:55.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 sudo[44435]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:55.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 sudo[44435]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:55 smithi103 sudo[49789]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:58:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:55 smithi103 sudo[49789]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:55 smithi103 sudo[49789]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:55 smithi103 sudo[49789]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:55 smithi103 ceph-mon[30398]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T01:58:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:55 smithi103 ceph-mon[30398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:58:55.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:55 smithi136 ceph-mon[35323]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T01:58:55.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:55.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:58:55.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:55 smithi136 ceph-mon[35323]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:58:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 sudo[44438]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:58:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 sudo[44438]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 sudo[44438]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:55 smithi172 sudo[44438]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:56.078 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:55 smithi136 sudo[43810]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:58:56.078 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:55 smithi136 sudo[43810]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:56.078 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:55 smithi136 sudo[43810]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:56.078 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:55 smithi136 sudo[43810]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:56.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 494 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:58:56.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:58:56.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:58:56.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 sudo[43813]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:58:56.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 sudo[43813]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:58:56.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 sudo[43813]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:58:56.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 sudo[43813]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:58:56.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 494 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:56.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:58:56.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:58:56.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:58:56.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:58:56.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:58:56.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:58:56.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:58:56.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:58:56.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:58:56.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:58:56.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:58:56.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:58:56.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 494 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:56.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:58:56.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:58:56.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:58:56.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:58:56.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:58:56.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:58:56.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:58:56.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:58:56.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:58:56.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:58:56.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:58:56.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:58:57.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:57 smithi103 ceph-mon[30398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:58:57.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:57 smithi103 ceph-mon[30398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:58:57.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:57 smithi103 ceph-mon[30398]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:57.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:57 smithi136 ceph-mon[35323]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:58:57.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:57 smithi136 ceph-mon[35323]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:58:57.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:57 smithi136 ceph-mon[35323]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:57.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:57 smithi172 ceph-mon[35340]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:58:57.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:57 smithi172 ceph-mon[35340]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:58:57.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:57 smithi172 ceph-mon[35340]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:59.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:58:59 smithi103 ceph-mon[30398]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:58:59 smithi136 ceph-mon[35323]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:58:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:58:59 smithi172 ceph-mon[35340]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:01.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:01 smithi103 ceph-mon[30398]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-16T01:59:01.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:01 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T01:59:01.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:01 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:01.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:01 smithi136 ceph-mon[35323]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-16T01:59:01.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:01 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T01:59:01.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:01 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:01.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:01 smithi172 ceph-mon[35340]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-16T01:59:01.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:01 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T01:59:01.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:01 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:02.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:02 smithi103 ceph-mon[30398]: Deploying daemon osd.3 on smithi103 2024-09-16T01:59:02.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:02 smithi136 ceph-mon[35323]: Deploying daemon osd.3 on smithi103 2024-09-16T01:59:02.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:02 smithi172 ceph-mon[35340]: Deploying daemon osd.3 on smithi103 2024-09-16T01:59:03.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:03 smithi103 ceph-mon[30398]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-16T01:59:03.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:03 smithi136 ceph-mon[35323]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-16T01:59:03.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:03 smithi172 ceph-mon[35340]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-16T01:59:05.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:05 smithi103 ceph-mon[30398]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-16T01:59:05.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:05 smithi136 ceph-mon[35323]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-16T01:59:05.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:05 smithi172 ceph-mon[35340]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-16T01:59:07.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:07 smithi103 ceph-mon[30398]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-16T01:59:07.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:07 smithi136 ceph-mon[35323]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-16T01:59:07.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:07 smithi172 ceph-mon[35340]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-16T01:59:08.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:08 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:08.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:08 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:08.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:08 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:08.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:08 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:08.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:08 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:08.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:08 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:08.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:08 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:08.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:08 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:08.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:08 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:09.730 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:09 smithi103 ceph-mon[30398]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-16T01:59:09.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:09 smithi136 ceph-mon[35323]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-16T01:59:09.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:09 smithi172 ceph-mon[35340]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-16T01:59:11.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:11 smithi103 ceph-mon[30398]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-16T01:59:11.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:11.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:11.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:11.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:11.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:11 smithi136 ceph-mon[35323]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-16T01:59:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:11 smithi172 ceph-mon[35340]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-16T01:59:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:12.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.794+0000 7f5aff400640 1 -- 172.21.15.103:0/1756342432 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b08062630 con 0x7f5aec0a31e0 2024-09-16T01:59:12.797 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 3 on host 'smithi103' 2024-09-16T01:59:12.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 -- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5aec0a31e0 msgr2=0x7f5aec0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:12.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5aec0a31e0 0x7f5aec0a56a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b0819b5e0 tx=0x7f5af4005f90 comp rx=0 tx=0).stop 2024-09-16T01:59:12.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 -- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b08074c50 msgr2=0x7f5b0819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:12.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b08074c50 0x7f5b0819a0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5af800e9c0 tx=0x7f5af800ee80 comp rx=0 tx=0).stop 2024-09-16T01:59:12.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 -- 172.21.15.103:0/1756342432 shutdown_connections 2024-09-16T01:59:12.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5aec0a31e0 0x7f5aec0a56a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:12.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b08109a80 0x7f5b081a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:12.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b08076220 0x7f5b0819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:12.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 --2- 172.21.15.103:0/1756342432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b08074c50 0x7f5b0819a0c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:12.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 -- 172.21.15.103:0/1756342432 >> 172.21.15.103:0/1756342432 conn(0x7f5b080fd390 msgr2=0x7f5b081059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:12.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 -- 172.21.15.103:0/1756342432 shutdown_connections 2024-09-16T01:59:12.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T01:59:12.797+0000 7f5b0d800640 1 -- 172.21.15.103:0/1756342432 wait complete. 2024-09-16T01:59:13.431 DEBUG:teuthology.orchestra.run.smithi103:osd.3> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T01:59:13.436 INFO:tasks.cephadm:Deploying osd.4 on smithi136 with /dev/vg_nvme/lv_3... 2024-09-16T01:59:13.436 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T01:59:13.691 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:59:13.736 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:13 smithi103 ceph-mon[30398]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:13.737 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:13.737 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:13.737 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:13.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:13 smithi136 ceph-mon[35323]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:13.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:13.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:13.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:13 smithi172 ceph-mon[35340]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:15.193 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 01:59:14 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3[51202]: 2024-09-16T01:59:14.929+0000 7fed4fc22640 -1 osd.3 0 log_to_monitors true 2024-09-16T01:59:15.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:15 smithi103 ceph-mon[30398]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:15.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:15 smithi103 ceph-mon[30398]: from='osd.3 [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:59:15.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:15 smithi103 ceph-mon[30398]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:59:15.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:15 smithi136 ceph-mon[35323]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:15.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:15 smithi136 ceph-mon[35323]: from='osd.3 [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:59:15.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:15 smithi136 ceph-mon[35323]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:15 smithi172 ceph-mon[35340]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:15 smithi172 ceph-mon[35340]: from='osd.3 [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:59:15.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:15 smithi172 ceph-mon[35340]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:59:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T01:59:15.916 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_3 2024-09-16T01:59:16.157 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:59:16.578 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:16 smithi136 ceph-mon[35323]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T01:59:16.578 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:16 smithi136 ceph-mon[35323]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T01:59:16.578 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:16.578 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:16 smithi136 ceph-mon[35323]: from='osd.3 [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T01:59:16.578 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:16 smithi136 ceph-mon[35323]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T01:59:16.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.576+0000 7f2766192640 1 -- 172.21.15.136:0/3341536387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27601026c0 msgr2=0x7f2760104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:16.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.576+0000 7f2766192640 1 --2- 172.21.15.136:0/3341536387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27601026c0 0x7f2760104ab0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2750009900 tx=0x7f275002a410 comp rx=0 tx=0).stop 2024-09-16T01:59:16.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.576+0000 7f2766192640 1 -- 172.21.15.136:0/3341536387 shutdown_connections 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.576+0000 7f2766192640 1 --2- 172.21.15.136:0/3341536387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2760107920 0x7f2760109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.576+0000 7f2766192640 1 --2- 172.21.15.136:0/3341536387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2760104ff0 0x7f27601073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.576+0000 7f2766192640 1 --2- 172.21.15.136:0/3341536387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27601026c0 0x7f2760104ab0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.576+0000 7f2766192640 1 -- 172.21.15.136:0/3341536387 >> 172.21.15.136:0/3341536387 conn(0x7f27600fc280 msgr2=0x7f27600fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.576+0000 7f2766192640 1 -- 172.21.15.136:0/3341536387 shutdown_connections 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.576+0000 7f2766192640 1 -- 172.21.15.136:0/3341536387 wait complete. 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.576+0000 7f2766192640 1 Processor -- start 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f2766192640 1 -- start start 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f2766192640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27601026c0 0x7f2760197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f2766192640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2760104ff0 0x7f27601984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f2766192640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2760107920 0x7f276019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f2766192640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f276010d900 con 0x7f2760104ff0 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f2766192640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f276010d780 con 0x7f27601026c0 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f2766192640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f276010da80 con 0x7f2760107920 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f275f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27601026c0 0x7f2760197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f275f7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27601026c0 0x7f2760197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:50506/0 (socket says 172.21.15.136:50506) 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f275f7fe640 1 -- 172.21.15.136:0/2654025839 learned_addr learned my addr 172.21.15.136:0/2654025839 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f2757fff640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2760104ff0 0x7f27601984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f275ffff640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2760107920 0x7f276019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f275f7fe640 1 -- 172.21.15.136:0/2654025839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2760107920 msgr2=0x7f276019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f275f7fe640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2760107920 0x7f276019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f275f7fe640 1 -- 172.21.15.136:0/2654025839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2760104ff0 msgr2=0x7f27601984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f275f7fe640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2760104ff0 0x7f27601984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.577+0000 7f275f7fe640 1 -- 172.21.15.136:0/2654025839 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276019fbe0 con 0x7f27601026c0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.578+0000 7f275f7fe640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27601026c0 0x7f2760197fb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2750009230 tx=0x7f2750004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.578+0000 7f275d7fa640 1 -- 172.21.15.136:0/2654025839 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275003f030 con 0x7f27601026c0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.578+0000 7f2766192640 1 -- 172.21.15.136:0/2654025839 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276019fe70 con 0x7f27601026c0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.578+0000 7f275d7fa640 1 -- 172.21.15.136:0/2654025839 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2750003ce0 con 0x7f27601026c0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.578+0000 7f275d7fa640 1 -- 172.21.15.136:0/2654025839 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750035070 con 0x7f27601026c0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.578+0000 7f2766192640 1 -- 172.21.15.136:0/2654025839 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27601a02f0 con 0x7f27601026c0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.579+0000 7f2766192640 1 -- 172.21.15.136:0/2654025839 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27600fdd70 con 0x7f27601026c0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.580+0000 7f275d7fa640 1 -- 172.21.15.136:0/2654025839 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2750042030 con 0x7f27601026c0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.581+0000 7f275d7fa640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f274c0a3270 0x7f274c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.581+0000 7f275d7fa640 1 -- 172.21.15.136:0/2654025839 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3389+0+0 (secure 0 0 0) 0x7f27500eef90 con 0x7f27601026c0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.581+0000 7f2757fff640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f274c0a3270 0x7f274c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.582+0000 7f2757fff640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f274c0a3270 0x7f274c0a5730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27601994d0 tx=0x7f274800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:16.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.588+0000 7f275d7fa640 1 -- 172.21.15.136:0/2654025839 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27500bc9a0 con 0x7f27601026c0 2024-09-16T01:59:16.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:16 smithi103 ceph-mon[30398]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T01:59:16.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:16 smithi103 ceph-mon[30398]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T01:59:16.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:16.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:16 smithi103 ceph-mon[30398]: from='osd.3 [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T01:59:16.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:16 smithi103 ceph-mon[30398]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T01:59:16.755 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 01:59:16 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3[51202]: 2024-09-16T01:59:16.508+0000 7fed49e00640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T01:59:16.755 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 01:59:16 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3[51202]: 2024-09-16T01:59:16.515+0000 7fed45800640 -1 osd.3 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:59:16.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:16.772+0000 7f2766192640 1 -- 172.21.15.136:0/2654025839 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2760062630 con 0x7f274c0a3270 2024-09-16T01:59:16.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:16 smithi172 ceph-mon[35340]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T01:59:16.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:16 smithi172 ceph-mon[35340]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T01:59:16.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:16.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:16 smithi172 ceph-mon[35340]: from='osd.3 [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T01:59:16.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:16 smithi172 ceph-mon[35340]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T01:59:17.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:17.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T01:59:17.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T01:59:17.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:17.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:17.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:59:17.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:59:17.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:17.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:17.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:17.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:17.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:17.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:17.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T01:59:17.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T01:59:17.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:17.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:17.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:59:17.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:59:17.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:17.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:17.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:17.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:17.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:17.775 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:17.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T01:59:17.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:17.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:17.797 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:17.798 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:18.588 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:18 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T01:59:18.588 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:18 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T01:59:18.588 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:18 smithi103 ceph-mon[30398]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:18.588 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:18 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi103 2024-09-16T01:59:18.588 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:18 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-16T01:59:18.588 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:18 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:18.588 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:18 smithi103 ceph-mon[30398]: osd.3 [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] boot 2024-09-16T01:59:18.588 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:18 smithi103 ceph-mon[30398]: osdmap e22: 4 total, 4 up, 4 in 2024-09-16T01:59:18.589 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:18 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:18.589 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:18 smithi103 ceph-mon[30398]: osdmap e23: 4 total, 4 up, 4 in 2024-09-16T01:59:18.631 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:18 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T01:59:18.631 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:18 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T01:59:18.631 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:18 smithi136 ceph-mon[35323]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:18.631 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:18 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi103 2024-09-16T01:59:18.631 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:18 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-16T01:59:18.631 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:18.631 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:18 smithi136 ceph-mon[35323]: osd.3 [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] boot 2024-09-16T01:59:18.631 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:18 smithi136 ceph-mon[35323]: osdmap e22: 4 total, 4 up, 4 in 2024-09-16T01:59:18.632 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:18 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:18.632 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:18 smithi136 ceph-mon[35323]: osdmap e23: 4 total, 4 up, 4 in 2024-09-16T01:59:18.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:18 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T01:59:18.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:18 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T01:59:18.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:18 smithi172 ceph-mon[35340]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:18.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:18 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi103 2024-09-16T01:59:18.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:18 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-16T01:59:18.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:18 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:18.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:18 smithi172 ceph-mon[35340]: osd.3 [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] boot 2024-09-16T01:59:18.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:18 smithi172 ceph-mon[35340]: osdmap e22: 4 total, 4 up, 4 in 2024-09-16T01:59:18.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:18 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:59:18.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:18 smithi172 ceph-mon[35340]: osdmap e23: 4 total, 4 up, 4 in 2024-09-16T01:59:19.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:19 smithi103 ceph-mon[30398]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:19.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:19 smithi103 ceph-mon[30398]: from='client.? 172.21.15.136:0/4137538295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35714991-3234-4719-b05f-604de810aac7"}]: dispatch 2024-09-16T01:59:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:19 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35714991-3234-4719-b05f-604de810aac7"}]: dispatch 2024-09-16T01:59:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:19 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35714991-3234-4719-b05f-604de810aac7"}]': finished 2024-09-16T01:59:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:19 smithi103 ceph-mon[30398]: osdmap e24: 5 total, 4 up, 5 in 2024-09-16T01:59:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:19 smithi136 ceph-mon[35323]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:19 smithi136 ceph-mon[35323]: from='client.? 172.21.15.136:0/4137538295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35714991-3234-4719-b05f-604de810aac7"}]: dispatch 2024-09-16T01:59:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:19 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35714991-3234-4719-b05f-604de810aac7"}]: dispatch 2024-09-16T01:59:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:19 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35714991-3234-4719-b05f-604de810aac7"}]': finished 2024-09-16T01:59:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:19 smithi136 ceph-mon[35323]: osdmap e24: 5 total, 4 up, 5 in 2024-09-16T01:59:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:19 smithi172 ceph-mon[35340]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:59:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:19 smithi172 ceph-mon[35340]: from='client.? 172.21.15.136:0/4137538295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35714991-3234-4719-b05f-604de810aac7"}]: dispatch 2024-09-16T01:59:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:19 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35714991-3234-4719-b05f-604de810aac7"}]: dispatch 2024-09-16T01:59:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:19 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35714991-3234-4719-b05f-604de810aac7"}]': finished 2024-09-16T01:59:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:19 smithi172 ceph-mon[35340]: osdmap e24: 5 total, 4 up, 5 in 2024-09-16T01:59:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:20.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:20 smithi103 ceph-mon[30398]: from='client.? 172.21.15.136:0/1040495546' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:59:20.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:20 smithi103 ceph-mon[30398]: osdmap e25: 5 total, 4 up, 5 in 2024-09-16T01:59:20.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:20 smithi136 ceph-mon[35323]: from='client.? 172.21.15.136:0/1040495546' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:59:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:20 smithi136 ceph-mon[35323]: osdmap e25: 5 total, 4 up, 5 in 2024-09-16T01:59:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:20 smithi172 ceph-mon[35340]: from='client.? 172.21.15.136:0/1040495546' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:59:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:20 smithi172 ceph-mon[35340]: osdmap e25: 5 total, 4 up, 5 in 2024-09-16T01:59:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:21 smithi136 ceph-mon[35323]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:21 smithi136 ceph-mon[35323]: osdmap e26: 5 total, 4 up, 5 in 2024-09-16T01:59:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:21 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:21.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:21 smithi172 ceph-mon[35340]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:21.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:21 smithi172 ceph-mon[35340]: osdmap e26: 5 total, 4 up, 5 in 2024-09-16T01:59:21.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:21 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:22.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:21 smithi103 ceph-mon[30398]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:22.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:21 smithi103 ceph-mon[30398]: osdmap e26: 5 total, 4 up, 5 in 2024-09-16T01:59:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:21 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:23.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:23 smithi136 ceph-mon[35323]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:23.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:23 smithi172 ceph-mon[35340]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:23 smithi103 ceph-mon[30398]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:25.736 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:25 smithi136 ceph-mon[35323]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:59:25.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:25 smithi172 ceph-mon[35340]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:59:26.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:25 smithi103 ceph-mon[30398]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T01:59:27.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:27 smithi136 ceph-mon[35323]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-16T01:59:27.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T01:59:27.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:27.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:27 smithi172 ceph-mon[35340]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-16T01:59:27.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T01:59:27.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:28.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:27 smithi103 ceph-mon[30398]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-16T01:59:28.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T01:59:28.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:28.768 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:28 smithi136 ceph-mon[35323]: Deploying daemon osd.4 on smithi136 2024-09-16T01:59:28.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:28 smithi172 ceph-mon[35340]: Deploying daemon osd.4 on smithi136 2024-09-16T01:59:29.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:28 smithi103 ceph-mon[30398]: Deploying daemon osd.4 on smithi136 2024-09-16T01:59:29.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:29 smithi172 ceph-mon[35340]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T01:59:29.814 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:29 smithi103 ceph-mon[30398]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T01:59:29.925 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:29 smithi136 ceph-mon[35323]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T01:59:31.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:31 smithi172 ceph-mon[35340]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:59:31.933 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:31 smithi136 ceph-mon[35323]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:59:32.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:31 smithi103 ceph-mon[30398]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:59:33.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:33 smithi172 ceph-mon[35340]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-16T01:59:33.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:33 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:33.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:33 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:33.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:33 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:33.838 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:33 smithi136 ceph-mon[35323]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-16T01:59:33.838 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:33 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:33.838 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:33 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:33.838 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:33 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:34.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:33 smithi103 ceph-mon[30398]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-16T01:59:34.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:33 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:34.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:33 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:34.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:33 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:35.801 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:35 smithi172 ceph-mon[35340]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:59:35.934 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:35 smithi136 ceph-mon[35323]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:59:36.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:35 smithi103 ceph-mon[30398]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:59:36.247 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.245+0000 7f275d7fa640 1 -- 172.21.15.136:0/2654025839 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2760062630 con 0x7f274c0a3270 2024-09-16T01:59:36.247 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 4 on host 'smithi136' 2024-09-16T01:59:36.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.247+0000 7f2766192640 1 -- 172.21.15.136:0/2654025839 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f274c0a3270 msgr2=0x7f274c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:36.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.247+0000 7f2766192640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f274c0a3270 0x7f274c0a5730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27601994d0 tx=0x7f274800b390 comp rx=0 tx=0).stop 2024-09-16T01:59:36.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.248+0000 7f2766192640 1 -- 172.21.15.136:0/2654025839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27601026c0 msgr2=0x7f2760197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:36.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.248+0000 7f2766192640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27601026c0 0x7f2760197fb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2750009230 tx=0x7f2750004360 comp rx=0 tx=0).stop 2024-09-16T01:59:36.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.248+0000 7f2766192640 1 -- 172.21.15.136:0/2654025839 shutdown_connections 2024-09-16T01:59:36.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.248+0000 7f2766192640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f274c0a3270 0x7f274c0a5730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:36.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.248+0000 7f2766192640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2760107920 0x7f276019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:36.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.248+0000 7f2766192640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2760104ff0 0x7f27601984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:36.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.248+0000 7f2766192640 1 --2- 172.21.15.136:0/2654025839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27601026c0 0x7f2760197fb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:36.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.248+0000 7f2766192640 1 -- 172.21.15.136:0/2654025839 >> 172.21.15.136:0/2654025839 conn(0x7f27600fc280 msgr2=0x7f2760102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:36.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.248+0000 7f2766192640 1 -- 172.21.15.136:0/2654025839 shutdown_connections 2024-09-16T01:59:36.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T01:59:36.248+0000 7f2766192640 1 -- 172.21.15.136:0/2654025839 wait complete. 2024-09-16T01:59:36.833 DEBUG:teuthology.orchestra.run.smithi136:osd.4> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T01:59:36.836 INFO:tasks.cephadm:Deploying osd.5 on smithi172 with /dev/vg_nvme/lv_3... 2024-09-16T01:59:36.836 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T01:59:37.073 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:36 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.073 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:36 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.073 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:36 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:37.074 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:36 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:37.074 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:36 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.074 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:36 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:37.074 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:36 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.074 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:36 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.074 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:36 smithi136 ceph-mon[35323]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:59:37.214 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:36 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.214 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:36 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.215 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:36 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:37.215 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:36 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:37.215 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:36 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.215 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:36 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:37.215 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:36 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.215 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:36 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.215 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:36 smithi172 ceph-mon[35340]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:37.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:36 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:36 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:36 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:37.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:36 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:37.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:36 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:36 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:37.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:36 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:36 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:37.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:36 smithi103 ceph-mon[30398]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:39.453 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 01:59:39 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4[46574]: 2024-09-16T01:59:39.377+0000 7feaa654d640 -1 osd.4 0 log_to_monitors true 2024-09-16T01:59:39.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:39 smithi103 ceph-mon[30398]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:39.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:39 smithi103 ceph-mon[30398]: from='osd.4 [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:59:39.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:39 smithi103 ceph-mon[30398]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:59:39.768 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:39 smithi172 ceph-mon[35340]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:39.768 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:39 smithi172 ceph-mon[35340]: from='osd.4 [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:59:39.769 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:39 smithi172 ceph-mon[35340]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:59:39.772 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T01:59:39.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:39 smithi136 ceph-mon[35323]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:39.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:39 smithi136 ceph-mon[35323]: from='osd.4 [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:59:39.777 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:39 smithi136 ceph-mon[35323]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:59:39.800 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_3 2024-09-16T01:59:40.045 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:59:40.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1318963990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d28102e00 msgr2=0x7f3d28103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:40.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 --2- 172.21.15.172:0/1318963990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d28102e00 0x7f3d28103280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009900 tx=0x7f3d1802a410 comp rx=0 tx=0).stop 2024-09-16T01:59:40.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1318963990 shutdown_connections 2024-09-16T01:59:40.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 --2- 172.21.15.172:0/1318963990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d281037c0 0x7f3d2810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:40.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 --2- 172.21.15.172:0/1318963990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d28102e00 0x7f3d28103280 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:40.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 --2- 172.21.15.172:0/1318963990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d28101c00 0x7f3d28102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:40.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1318963990 >> 172.21.15.172:0/1318963990 conn(0x7f3d280fd390 msgr2=0x7f3d280ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:40.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1318963990 shutdown_connections 2024-09-16T01:59:40.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1318963990 wait complete. 2024-09-16T01:59:40.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 Processor -- start 2024-09-16T01:59:40.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 -- start start 2024-09-16T01:59:40.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d28101c00 0x7f3d28197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:40.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d28102e00 0x7f3d28198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:40.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d281037c0 0x7f3d2819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:40.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2810d880 con 0x7f3d28102e00 2024-09-16T01:59:40.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2810d700 con 0x7f3d281037c0 2024-09-16T01:59:40.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.472+0000 7f3d2f4bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2810da00 con 0x7f3d28101c00 2024-09-16T01:59:40.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2d257640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d28101c00 0x7f3d28197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:40.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2d257640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d28101c00 0x7f3d28197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47516/0 (socket says 172.21.15.172:47516) 2024-09-16T01:59:40.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2d257640 1 -- 172.21.15.172:0/1588473612 learned_addr learned my addr 172.21.15.172:0/1588473612 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T01:59:40.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2da58640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d281037c0 0x7f3d2819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:40.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2ca56640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d28102e00 0x7f3d28198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:40.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2d257640 1 -- 172.21.15.172:0/1588473612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d281037c0 msgr2=0x7f3d2819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:40.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2d257640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d281037c0 0x7f3d2819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:40.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2d257640 1 -- 172.21.15.172:0/1588473612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d28102e00 msgr2=0x7f3d28198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:40.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2d257640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d28102e00 0x7f3d28198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:40.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2d257640 1 -- 172.21.15.172:0/1588473612 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2819fb10 con 0x7f3d28101c00 2024-09-16T01:59:40.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2d257640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d28101c00 0x7f3d28197ee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400e9c0 tx=0x7f3d2400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:40.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d1e7fc640 1 -- 172.21.15.172:0/1588473612 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400cd80 con 0x7f3d28101c00 2024-09-16T01:59:40.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1588473612 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2819fe00 con 0x7f3d28101c00 2024-09-16T01:59:40.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d1e7fc640 1 -- 172.21.15.172:0/1588473612 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d240035f0 con 0x7f3d28101c00 2024-09-16T01:59:40.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d1e7fc640 1 -- 172.21.15.172:0/1588473612 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24010690 con 0x7f3d28101c00 2024-09-16T01:59:40.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.473+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1588473612 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281a0200 con 0x7f3d28101c00 2024-09-16T01:59:40.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.474+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1588473612 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d28109020 con 0x7f3d28101c00 2024-09-16T01:59:40.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.475+0000 7f3d1e7fc640 1 -- 172.21.15.172:0/1588473612 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3d240037a0 con 0x7f3d28101c00 2024-09-16T01:59:40.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.476+0000 7f3d1e7fc640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3d140a3270 0x7f3d140a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:40.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.476+0000 7f3d1e7fc640 1 -- 172.21.15.172:0/1588473612 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3821+0+0 (secure 0 0 0) 0x7f3d240cec00 con 0x7f3d28101c00 2024-09-16T01:59:40.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.476+0000 7f3d2ca56640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3d140a3270 0x7f3d140a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:40.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.477+0000 7f3d2ca56640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3d140a3270 0x7f3d140a5730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d18000c00 tx=0x7f3d180315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:40.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.479+0000 7f3d1e7fc640 1 -- 172.21.15.172:0/1588473612 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d2409cab0 con 0x7f3d28101c00 2024-09-16T01:59:40.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T01:59:40.657+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1588473612 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3d28062630 con 0x7f3d140a3270 2024-09-16T01:59:40.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T01:59:40.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='osd.4 [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:59:40.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T01:59:40.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:40.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:59:40.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi136 2024-09-16T01:59:40.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:40.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:40.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi136 to 8544M 2024-09-16T01:59:40.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:40.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:40.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:40 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T01:59:40.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='osd.4 [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:59:40.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T01:59:40.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:40.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:59:40.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi136 2024-09-16T01:59:40.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:40.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:40.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi136 to 8544M 2024-09-16T01:59:40.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:40.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:40.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.774 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4[46574]: 2024-09-16T01:59:40.467+0000 7feaa1cad640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T01:59:40.775 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 01:59:40 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4[46574]: 2024-09-16T01:59:40.475+0000 7fea9e2e7640 -1 osd.4 28 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:59:40.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T01:59:40.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='osd.4 [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:59:40.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T01:59:40.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi136 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi136 to 8544M 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:40.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:40 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:41.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:41 smithi172 ceph-mon[35340]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:41.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:41 smithi172 ceph-mon[35340]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T01:59:41.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:41 smithi172 ceph-mon[35340]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T01:59:41.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:41.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:41.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:59:41.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:59:41.728 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:41.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:41 smithi103 ceph-mon[30398]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:41 smithi103 ceph-mon[30398]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T01:59:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:41 smithi103 ceph-mon[30398]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T01:59:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:59:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:59:41.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:41 smithi136 ceph-mon[35323]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:41.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:41 smithi136 ceph-mon[35323]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T01:59:41.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:41 smithi136 ceph-mon[35323]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T01:59:41.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:41.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:41.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:59:41.775 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:59:41.775 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:42.646 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:42 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T01:59:42.646 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:42 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T01:59:42.646 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:42 smithi172 ceph-mon[35340]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:42.647 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:42 smithi172 ceph-mon[35340]: osd.4 [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] boot 2024-09-16T01:59:42.647 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:42 smithi172 ceph-mon[35340]: osdmap e29: 5 total, 5 up, 5 in 2024-09-16T01:59:42.647 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:42 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:42.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:42 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T01:59:42.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:42 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T01:59:42.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:42 smithi103 ceph-mon[30398]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:42.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:42 smithi103 ceph-mon[30398]: osd.4 [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] boot 2024-09-16T01:59:42.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:42 smithi103 ceph-mon[30398]: osdmap e29: 5 total, 5 up, 5 in 2024-09-16T01:59:42.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:42 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:42.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:42 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T01:59:42.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:42 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T01:59:42.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:42 smithi136 ceph-mon[35323]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:42.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:42 smithi136 ceph-mon[35323]: osd.4 [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] boot 2024-09-16T01:59:42.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:42 smithi136 ceph-mon[35323]: osdmap e29: 5 total, 5 up, 5 in 2024-09-16T01:59:42.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:42 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:59:43.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:43 smithi103 ceph-mon[30398]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:43.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:43 smithi103 ceph-mon[30398]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T01:59:43.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:43 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/3511327806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17f3197-8dc3-4e97-b7ee-a36975327aa7"}]: dispatch 2024-09-16T01:59:43.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:43 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17f3197-8dc3-4e97-b7ee-a36975327aa7"}]: dispatch 2024-09-16T01:59:43.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:43 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f17f3197-8dc3-4e97-b7ee-a36975327aa7"}]': finished 2024-09-16T01:59:43.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:43 smithi103 ceph-mon[30398]: osdmap e31: 6 total, 5 up, 6 in 2024-09-16T01:59:43.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:43 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:59:43.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:43 smithi136 ceph-mon[35323]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:43.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:43 smithi136 ceph-mon[35323]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T01:59:43.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:43 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/3511327806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17f3197-8dc3-4e97-b7ee-a36975327aa7"}]: dispatch 2024-09-16T01:59:43.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:43 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17f3197-8dc3-4e97-b7ee-a36975327aa7"}]: dispatch 2024-09-16T01:59:43.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:43 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f17f3197-8dc3-4e97-b7ee-a36975327aa7"}]': finished 2024-09-16T01:59:43.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:43 smithi136 ceph-mon[35323]: osdmap e31: 6 total, 5 up, 6 in 2024-09-16T01:59:43.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:43 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:59:43.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:43 smithi172 ceph-mon[35340]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:59:43.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:43 smithi172 ceph-mon[35340]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T01:59:43.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:43 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/3511327806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17f3197-8dc3-4e97-b7ee-a36975327aa7"}]: dispatch 2024-09-16T01:59:43.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:43 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17f3197-8dc3-4e97-b7ee-a36975327aa7"}]: dispatch 2024-09-16T01:59:43.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:43 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f17f3197-8dc3-4e97-b7ee-a36975327aa7"}]': finished 2024-09-16T01:59:43.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:43 smithi172 ceph-mon[35340]: osdmap e31: 6 total, 5 up, 6 in 2024-09-16T01:59:43.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:43 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:59:44.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:44 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/134933519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:59:44.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:44 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/134933519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:59:44.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:44 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/134933519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:59:45.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:45 smithi103 ceph-mon[30398]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:45.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:45 smithi136 ceph-mon[35323]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:45.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:45 smithi172 ceph-mon[35340]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:47 smithi103 ceph-mon[30398]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:47.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:47 smithi136 ceph-mon[35323]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:47 smithi172 ceph-mon[35340]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:49.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:49 smithi103 ceph-mon[30398]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:49 smithi136 ceph-mon[35323]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:49.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:49 smithi172 ceph-mon[35340]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:51.641 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:51 smithi172 ceph-mon[35340]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:51.826 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:51 smithi136 ceph-mon[35323]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:52.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:51 smithi103 ceph-mon[30398]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:52.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:52 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T01:59:52.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:52 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:52.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:52 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T01:59:52.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:52 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:53.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:52 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T01:59:53.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:52 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:53.616 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:53 smithi172 ceph-mon[35340]: Deploying daemon osd.5 on smithi172 2024-09-16T01:59:53.616 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:53 smithi172 ceph-mon[35340]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:53 smithi136 ceph-mon[35323]: Deploying daemon osd.5 on smithi172 2024-09-16T01:59:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:53 smithi136 ceph-mon[35323]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:54.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:53 smithi103 ceph-mon[30398]: Deploying daemon osd.5 on smithi172 2024-09-16T01:59:54.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:53 smithi103 ceph-mon[30398]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:55.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:55 smithi136 ceph-mon[35323]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:55 smithi172 ceph-mon[35340]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:56.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:55 smithi103 ceph-mon[30398]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:57.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:57 smithi172 ceph-mon[35340]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:58.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:57 smithi103 ceph-mon[30398]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:58.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:57 smithi136 ceph-mon[35323]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:59:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:58.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:59.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:59.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:59.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:59.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:59.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:59.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T01:59:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 01:59:59 smithi172 ceph-mon[35340]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:00.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 01:59:59 smithi103 ceph-mon[30398]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:00.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 01:59:59 smithi136 ceph-mon[35323]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:00.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:00 smithi172 ceph-mon[35340]: overall HEALTH_OK 2024-09-16T02:00:00.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:00.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:00.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:00.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:00.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:01.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:00 smithi103 ceph-mon[30398]: overall HEALTH_OK 2024-09-16T02:00:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:00 smithi136 ceph-mon[35323]: overall HEALTH_OK 2024-09-16T02:00:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:01.655 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:01 smithi172 ceph-mon[35340]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:01.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.701+0000 7f3d1e7fc640 1 -- 172.21.15.172:0/1588473612 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3d28062630 con 0x7f3d140a3270 2024-09-16T02:00:01.703 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 5 on host 'smithi172' 2024-09-16T02:00:01.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1588473612 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3d140a3270 msgr2=0x7f3d140a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3d140a3270 0x7f3d140a5730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d18000c00 tx=0x7f3d180315d0 comp rx=0 tx=0).stop 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1588473612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d28101c00 msgr2=0x7f3d28197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d28101c00 0x7f3d28197ee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400e9c0 tx=0x7f3d2400ee80 comp rx=0 tx=0).stop 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1588473612 shutdown_connections 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3d140a3270 0x7f3d140a5730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d281037c0 0x7f3d2819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d28102e00 0x7f3d28198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 --2- 172.21.15.172:0/1588473612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d28101c00 0x7f3d28197ee0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1588473612 >> 172.21.15.172:0/1588473612 conn(0x7f3d280fd390 msgr2=0x7f3d281061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1588473612 shutdown_connections 2024-09-16T02:00:01.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:01.703+0000 7f3d2f4bb640 1 -- 172.21.15.172:0/1588473612 wait complete. 2024-09-16T02:00:02.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:01 smithi103 ceph-mon[30398]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:02.022 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:01 smithi136 ceph-mon[35323]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:02.303 DEBUG:teuthology.orchestra.run.smithi172:osd.5> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T02:00:02.305 INFO:tasks.cephadm:Deploying osd.6 on smithi103 with /dev/vg_nvme/lv_2... 2024-09-16T02:00:02.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T02:00:02.685 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:02.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:02 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:02.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:02 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:02.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:02 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:02.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:02.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:02.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:03.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:02 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:03.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:02 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:03.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:02 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:03.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:03 smithi172 ceph-mon[35340]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:03.795 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:00:03 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:00:03.774+0000 7f9daae9b640 -1 osd.5 0 log_to_monitors true 2024-09-16T02:00:03.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:03 smithi103 ceph-mon[30398]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:04.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:03 smithi136 ceph-mon[35323]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:04.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:04 smithi103 ceph-mon[30398]: from='osd.5 [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T02:00:04.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:04 smithi103 ceph-mon[30398]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T02:00:04.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:04 smithi172 ceph-mon[35340]: from='osd.5 [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T02:00:04.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:04 smithi172 ceph-mon[35340]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T02:00:05.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:04 smithi136 ceph-mon[35323]: from='osd.5 [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T02:00:05.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:04 smithi136 ceph-mon[35323]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T02:00:05.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:05.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T02:00:05.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: osdmap e32: 6 total, 5 up, 6 in 2024-09-16T02:00:05.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:05.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='osd.5 [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:00:05.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:00:05.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:05.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:05.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:05.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:05.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:05.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:05.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:05.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:05 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:05.665 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:00:05.700 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-09-16T02:00:05.795 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:00:05.555+0000 7f9da65fb640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T02:00:05.795 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:00:05.561+0000 7f9da2c35640 -1 osd.5 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T02:00:05.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:05.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T02:00:05.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: osdmap e32: 6 total, 5 up, 6 in 2024-09-16T02:00:05.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:05.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='osd.5 [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:00:05.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:00:05.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:05.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:05.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:05.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:05.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:05.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:05.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:05.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:05 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:06.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:06.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T02:00:06.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: osdmap e32: 6 total, 5 up, 6 in 2024-09-16T02:00:06.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:06.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='osd.5 [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:00:06.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:00:06.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:06.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:06.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:06.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:06.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:06.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:06.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:06.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:05 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:06.086 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:06.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.661+0000 7f8d61000640 1 -- 172.21.15.103:0/1469397020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d5c076220 msgr2=0x7f8d5c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:06.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.661+0000 7f8d61000640 1 --2- 172.21.15.103:0/1469397020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d5c076220 0x7f8d5c074680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009900 tx=0x7f8d4c02a3f0 comp rx=0 tx=0).stop 2024-09-16T02:00:06.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.661+0000 7f8d61000640 1 -- 172.21.15.103:0/1469397020 shutdown_connections 2024-09-16T02:00:06.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.661+0000 7f8d61000640 1 --2- 172.21.15.103:0/1469397020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d5c10e020 0x7f8d5c110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:06.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.661+0000 7f8d61000640 1 --2- 172.21.15.103:0/1469397020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d5c074c50 0x7f8d5c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:06.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.661+0000 7f8d61000640 1 --2- 172.21.15.103:0/1469397020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d5c076220 0x7f8d5c074680 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:06.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.661+0000 7f8d61000640 1 -- 172.21.15.103:0/1469397020 >> 172.21.15.103:0/1469397020 conn(0x7f8d5c0fd1f0 msgr2=0x7f8d5c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:06.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.661+0000 7f8d61000640 1 -- 172.21.15.103:0/1469397020 shutdown_connections 2024-09-16T02:00:06.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.661+0000 7f8d61000640 1 -- 172.21.15.103:0/1469397020 wait complete. 2024-09-16T02:00:06.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.662+0000 7f8d61000640 1 Processor -- start 2024-09-16T02:00:06.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.662+0000 7f8d61000640 1 -- start start 2024-09-16T02:00:06.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d61000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d5c074c50 0x7f8d5c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:06.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d61000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d5c076220 0x7f8d5c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d61000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d5c10e020 0x7f8d5c1a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d61000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c113ce0 con 0x7f8d5c074c50 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d61000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c113b60 con 0x7f8d5c10e020 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d61000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c113e60 con 0x7f8d5c076220 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d5b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d5c074c50 0x7f8d5c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d5b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d5c074c50 0x7f8d5c19e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36376/0 (socket says 172.21.15.103:36376) 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d5b400640 1 -- 172.21.15.103:0/372520698 learned_addr learned my addr 172.21.15.103:0/372520698 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d5aa00640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d5c076220 0x7f8d5c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d5be00640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d5c10e020 0x7f8d5c1a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d5aa00640 1 -- 172.21.15.103:0/372520698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d5c10e020 msgr2=0x7f8d5c1a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d5aa00640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d5c10e020 0x7f8d5c1a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d5aa00640 1 -- 172.21.15.103:0/372520698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d5c074c50 msgr2=0x7f8d5c19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:06.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d5aa00640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d5c074c50 0x7f8d5c19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:06.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.663+0000 7f8d5aa00640 1 -- 172.21.15.103:0/372520698 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c1a5f50 con 0x7f8d5c076220 2024-09-16T02:00:06.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.664+0000 7f8d5aa00640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d5c076220 0x7f8d5c19e860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800e9f0 tx=0x7f8d4800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:06.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.664+0000 7f8d53e00640 1 -- 172.21.15.103:0/372520698 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800ccc0 con 0x7f8d5c076220 2024-09-16T02:00:06.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.664+0000 7f8d53e00640 1 -- 172.21.15.103:0/372520698 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d48010910 con 0x7f8d5c076220 2024-09-16T02:00:06.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.664+0000 7f8d53e00640 1 -- 172.21.15.103:0/372520698 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48018960 con 0x7f8d5c076220 2024-09-16T02:00:06.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.664+0000 7f8d61000640 1 -- 172.21.15.103:0/372520698 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5c1a6240 con 0x7f8d5c076220 2024-09-16T02:00:06.668 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:06 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi172 2024-09-16T02:00:06.668 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:06 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-16T02:00:06.668 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:06 smithi103 ceph-mon[30398]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T02:00:06.668 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:06 smithi103 ceph-mon[30398]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T02:00:06.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:06 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:06.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:06 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:06.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.664+0000 7f8d61000640 1 -- 172.21.15.103:0/372520698 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5c1a67d0 con 0x7f8d5c076220 2024-09-16T02:00:06.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.665+0000 7f8d61000640 1 -- 172.21.15.103:0/372520698 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5c1039c0 con 0x7f8d5c076220 2024-09-16T02:00:06.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.670+0000 7f8d53e00640 1 -- 172.21.15.103:0/372520698 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8d48010ac0 con 0x7f8d5c076220 2024-09-16T02:00:06.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.671+0000 7f8d53e00640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8d400a32f0 0x7f8d400a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:06.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.672+0000 7f8d53e00640 1 -- 172.21.15.103:0/372520698 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4566+0+0 (secure 0 0 0) 0x7f8d480cef70 con 0x7f8d5c076220 2024-09-16T02:00:06.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.672+0000 7f8d5b400640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8d400a32f0 0x7f8d400a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:06.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.672+0000 7f8d53e00640 1 -- 172.21.15.103:0/372520698 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d480cf3f0 con 0x7f8d5c076220 2024-09-16T02:00:06.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.672+0000 7f8d5b400640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8d400a32f0 0x7f8d400a57b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c000c00 tx=0x7f8d4c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:06.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:06.858+0000 7f8d61000640 1 -- 172.21.15.103:0/372520698 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8d5c062630 con 0x7f8d400a32f0 2024-09-16T02:00:07.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:06 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi172 2024-09-16T02:00:07.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:06 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-16T02:00:07.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:06 smithi136 ceph-mon[35323]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T02:00:07.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:06 smithi136 ceph-mon[35323]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T02:00:07.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:06 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:07.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:06 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:07.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:06 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi172 2024-09-16T02:00:07.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:06 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-16T02:00:07.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:06 smithi172 ceph-mon[35340]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T02:00:07.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:06 smithi172 ceph-mon[35340]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T02:00:07.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:06 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:07.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:06 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:07.716 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:07 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T02:00:07.716 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:07 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T02:00:07.716 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:07 smithi103 ceph-mon[30398]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:07.716 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:07.716 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:07 smithi103 ceph-mon[30398]: osd.5 [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] boot 2024-09-16T02:00:07.717 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:07 smithi103 ceph-mon[30398]: osdmap e34: 6 total, 6 up, 6 in 2024-09-16T02:00:07.717 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:07.717 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:00:07.717 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:00:07.717 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:07.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:07 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T02:00:07.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:07 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T02:00:07.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:07 smithi172 ceph-mon[35340]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:07.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:07.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:07 smithi172 ceph-mon[35340]: osd.5 [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] boot 2024-09-16T02:00:07.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:07 smithi172 ceph-mon[35340]: osdmap e34: 6 total, 6 up, 6 in 2024-09-16T02:00:07.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:07.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:00:07.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:00:07.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:08.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:07 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T02:00:08.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:07 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T02:00:08.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:07 smithi136 ceph-mon[35323]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T02:00:08.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:08.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:07 smithi136 ceph-mon[35323]: osd.5 [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] boot 2024-09-16T02:00:08.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:07 smithi136 ceph-mon[35323]: osdmap e34: 6 total, 6 up, 6 in 2024-09-16T02:00:08.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T02:00:08.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:00:08.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:00:08.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:08.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:08 smithi103 ceph-mon[30398]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:08.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:08 smithi103 ceph-mon[30398]: osdmap e35: 6 total, 6 up, 6 in 2024-09-16T02:00:09.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:08 smithi136 ceph-mon[35323]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:09.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:08 smithi136 ceph-mon[35323]: osdmap e35: 6 total, 6 up, 6 in 2024-09-16T02:00:09.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:08 smithi172 ceph-mon[35340]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:09.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:08 smithi172 ceph-mon[35340]: osdmap e35: 6 total, 6 up, 6 in 2024-09-16T02:00:09.924 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:09 smithi136 ceph-mon[35323]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:09.924 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:09 smithi136 ceph-mon[35323]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T02:00:09.975 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:09 smithi103 ceph-mon[30398]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:09.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:09 smithi103 ceph-mon[30398]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T02:00:10.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:09 smithi172 ceph-mon[35340]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:10.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:09 smithi172 ceph-mon[35340]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T02:00:10.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:10 smithi103 ceph-mon[30398]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T02:00:10.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:10 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3966653141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf"}]: dispatch 2024-09-16T02:00:10.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:10 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf"}]: dispatch 2024-09-16T02:00:10.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:10 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf"}]': finished 2024-09-16T02:00:10.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:10 smithi103 ceph-mon[30398]: osdmap e38: 7 total, 6 up, 7 in 2024-09-16T02:00:10.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:10 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:10.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:10 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1600067416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:00:11.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:10 smithi136 ceph-mon[35323]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T02:00:11.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:10 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3966653141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf"}]: dispatch 2024-09-16T02:00:11.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:10 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf"}]: dispatch 2024-09-16T02:00:11.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:10 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf"}]': finished 2024-09-16T02:00:11.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:10 smithi136 ceph-mon[35323]: osdmap e38: 7 total, 6 up, 7 in 2024-09-16T02:00:11.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:10 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:11.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:10 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1600067416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:00:11.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:10 smithi172 ceph-mon[35340]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T02:00:11.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:10 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3966653141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf"}]: dispatch 2024-09-16T02:00:11.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:10 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf"}]: dispatch 2024-09-16T02:00:11.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:10 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf"}]': finished 2024-09-16T02:00:11.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:10 smithi172 ceph-mon[35340]: osdmap e38: 7 total, 6 up, 7 in 2024-09-16T02:00:11.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:10 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:11.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:10 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1600067416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:00:12.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:11 smithi103 ceph-mon[30398]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-09-16T02:00:12.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:11 smithi136 ceph-mon[35323]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-09-16T02:00:12.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:11 smithi172 ceph-mon[35340]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-09-16T02:00:14.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:13 smithi103 ceph-mon[30398]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-16T02:00:14.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:13 smithi136 ceph-mon[35323]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-16T02:00:14.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:13 smithi172 ceph-mon[35340]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-16T02:00:15.694 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:15 smithi103 ceph-mon[30398]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T02:00:16.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:15 smithi136 ceph-mon[35323]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T02:00:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:15 smithi172 ceph-mon[35340]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T02:00:17.946 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:17 smithi103 ceph-mon[30398]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T02:00:18.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:17 smithi136 ceph-mon[35323]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T02:00:18.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:17 smithi172 ceph-mon[35340]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T02:00:19.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:19 smithi103 ceph-mon[30398]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:19.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T02:00:19.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:20.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:19 smithi136 ceph-mon[35323]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:20.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T02:00:20.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:19 smithi172 ceph-mon[35340]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T02:00:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:20.974 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:20 smithi103 ceph-mon[30398]: Deploying daemon osd.6 on smithi103 2024-09-16T02:00:21.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:20 smithi136 ceph-mon[35323]: Deploying daemon osd.6 on smithi103 2024-09-16T02:00:21.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:20 smithi172 ceph-mon[35340]: Deploying daemon osd.6 on smithi103 2024-09-16T02:00:21.703 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:21 smithi103 ceph-mon[30398]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:22.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:21 smithi136 ceph-mon[35323]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:22.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:21 smithi172 ceph-mon[35340]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:23.944 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:23 smithi103 ceph-mon[30398]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:23 smithi136 ceph-mon[35323]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:23 smithi172 ceph-mon[35340]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:25.964 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:25 smithi103 ceph-mon[30398]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:25 smithi136 ceph-mon[35323]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:25 smithi172 ceph-mon[35340]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:27.949 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:27 smithi103 ceph-mon[30398]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:27.950 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:27.950 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:27.950 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:27 smithi136 ceph-mon[35323]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:27 smithi172 ceph-mon[35340]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:30.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:29 smithi103 ceph-mon[30398]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:29 smithi136 ceph-mon[35323]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:30.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:29 smithi172 ceph-mon[35340]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.886+0000 7f8d53e00640 1 -- 172.21.15.103:0/372520698 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8d5c062630 con 0x7f8d400a32f0 2024-09-16T02:00:30.889 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 6 on host 'smithi103' 2024-09-16T02:00:30.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 -- 172.21.15.103:0/372520698 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8d400a32f0 msgr2=0x7f8d400a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:30.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8d400a32f0 0x7f8d400a57b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c000c00 tx=0x7f8d4c0046c0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 -- 172.21.15.103:0/372520698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d5c076220 msgr2=0x7f8d5c19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:30.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d5c076220 0x7f8d5c19e860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800e9f0 tx=0x7f8d4800eeb0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 -- 172.21.15.103:0/372520698 shutdown_connections 2024-09-16T02:00:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8d400a32f0 0x7f8d400a57b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d5c10e020 0x7f8d5c1a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d5c076220 0x7f8d5c19e860 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 --2- 172.21.15.103:0/372520698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d5c074c50 0x7f8d5c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 -- 172.21.15.103:0/372520698 >> 172.21.15.103:0/372520698 conn(0x7f8d5c0fd1f0 msgr2=0x7f8d5c0ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 -- 172.21.15.103:0/372520698 shutdown_connections 2024-09-16T02:00:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:00:30.889+0000 7f8d61000640 1 -- 172.21.15.103:0/372520698 wait complete. 2024-09-16T02:00:31.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:31.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:31.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:30 smithi103 ceph-mon[30398]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:30 smithi136 ceph-mon[35323]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:30 smithi172 ceph-mon[35340]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:31.616 DEBUG:teuthology.orchestra.run.smithi103:osd.6> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T02:00:31.620 INFO:tasks.cephadm:Deploying osd.7 on smithi136 with /dev/vg_nvme/lv_2... 2024-09-16T02:00:31.620 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T02:00:31.830 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:31 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:31.830 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:31 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.830 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:31 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:31.877 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:00:31.999 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:31 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:32.000 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:31 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:32.000 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:31 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:31 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:31 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:31 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:32.819 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:32 smithi103 ceph-mon[30398]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:33.009 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:32 smithi136 ceph-mon[35323]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:33.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:32 smithi172 ceph-mon[35340]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:33.381 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:00:33 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:00:33.110+0000 7f3649435640 -1 osd.6 0 log_to_monitors true 2024-09-16T02:00:34.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:33 smithi103 ceph-mon[30398]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T02:00:34.008 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:33 smithi136 ceph-mon[35323]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T02:00:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:33 smithi172 ceph-mon[35340]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T02:00:34.264 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T02:00:34.291 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_2 2024-09-16T02:00:34.540 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:00:34.914 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:34 smithi136 ceph-mon[35323]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T02:00:34.914 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:34 smithi136 ceph-mon[35323]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T02:00:34.914 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:34 smithi136 ceph-mon[35323]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T02:00:34.914 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:34 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:34.914 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:34 smithi136 ceph-mon[35323]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:35.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:34.999+0000 7f2bc046e640 1 -- 172.21.15.136:0/1887269465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb8102e20 msgr2=0x7f2bb81032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:35.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:34.999+0000 7f2bc046e640 1 --2- 172.21.15.136:0/1887269465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb8102e20 0x7f2bb81032a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009900 tx=0x7f2ba802a410 comp rx=0 tx=0).stop 2024-09-16T02:00:35.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:34.999+0000 7f2bc046e640 1 -- 172.21.15.136:0/1887269465 shutdown_connections 2024-09-16T02:00:35.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:34.999+0000 7f2bc046e640 1 --2- 172.21.15.136:0/1887269465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bb81037e0 0x7f2bb810a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:35.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:34.999+0000 7f2bc046e640 1 --2- 172.21.15.136:0/1887269465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb8102e20 0x7f2bb81032a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:35.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:34.999+0000 7f2bc046e640 1 --2- 172.21.15.136:0/1887269465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bb8101c20 0x7f2bb8102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:34.999+0000 7f2bc046e640 1 -- 172.21.15.136:0/1887269465 >> 172.21.15.136:0/1887269465 conn(0x7f2bb80fd3b0 msgr2=0x7f2bb80ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:34.999+0000 7f2bc046e640 1 -- 172.21.15.136:0/1887269465 shutdown_connections 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:34.999+0000 7f2bc046e640 1 -- 172.21.15.136:0/1887269465 wait complete. 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bc046e640 1 Processor -- start 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bc046e640 1 -- start start 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bc046e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bb8101c20 0x7f2bb8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bc046e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bb8102e20 0x7f2bb80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bc046e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb81037e0 0x7f2bb8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bc046e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb810d900 con 0x7f2bb8102e20 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bc046e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb810d780 con 0x7f2bb81037e0 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bc046e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb810da80 con 0x7f2bb8101c20 2024-09-16T02:00:35.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bbea0b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb81037e0 0x7f2bb8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bbea0b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb81037e0 0x7f2bb8074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:37738/0 (socket says 172.21.15.136:37738) 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.000+0000 7f2bbea0b640 1 -- 172.21.15.136:0/1270662805 learned_addr learned my addr 172.21.15.136:0/1270662805 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2bbe20a640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bb8101c20 0x7f2bb8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2bbea0b640 1 -- 172.21.15.136:0/1270662805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bb8101c20 msgr2=0x7f2bb8075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2bbea0b640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bb8101c20 0x7f2bb8075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2bbea0b640 1 -- 172.21.15.136:0/1270662805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bb8102e20 msgr2=0x7f2bb80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2bbea0b640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bb8102e20 0x7f2bb80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2bbea0b640 1 -- 172.21.15.136:0/1270662805 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb80753e0 con 0x7f2bb81037e0 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2bbe20a640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bb8101c20 0x7f2bb8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2bbea0b640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb81037e0 0x7f2bb8074b20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000b970 tx=0x7f2bb000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:35.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2baf7fe640 1 -- 172.21.15.136:0/1270662805 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0019070 con 0x7f2bb81037e0 2024-09-16T02:00:35.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2baf7fe640 1 -- 172.21.15.136:0/1270662805 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb000cb40 con 0x7f2bb81037e0 2024-09-16T02:00:35.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2bc046e640 1 -- 172.21.15.136:0/1270662805 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb81a2050 con 0x7f2bb81037e0 2024-09-16T02:00:35.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2baf7fe640 1 -- 172.21.15.136:0/1270662805 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0007650 con 0x7f2bb81037e0 2024-09-16T02:00:35.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.001+0000 7f2bc046e640 1 -- 172.21.15.136:0/1270662805 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb81a2590 con 0x7f2bb81037e0 2024-09-16T02:00:35.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:34 smithi103 ceph-mon[30398]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T02:00:35.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:34 smithi103 ceph-mon[30398]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T02:00:35.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:34 smithi103 ceph-mon[30398]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T02:00:35.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:34 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:35.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:34 smithi103 ceph-mon[30398]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:35.006 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:00:34 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:00:34.716+0000 7f3643200640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T02:00:35.006 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:00:34 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:00:34.725+0000 7f363ec00640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T02:00:35.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.002+0000 7f2bc046e640 1 -- 172.21.15.136:0/1270662805 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb8102020 con 0x7f2bb81037e0 2024-09-16T02:00:35.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.004+0000 7f2baf7fe640 1 -- 172.21.15.136:0/1270662805 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2bb001c070 con 0x7f2bb81037e0 2024-09-16T02:00:35.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.004+0000 7f2baf7fe640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ba40a3240 0x7f2ba40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:35.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.005+0000 7f2baf7fe640 1 -- 172.21.15.136:0/1270662805 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4685+0+0 (secure 0 0 0) 0x7f2bb00ce5a0 con 0x7f2bb81037e0 2024-09-16T02:00:35.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.005+0000 7f2bbe20a640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ba40a3240 0x7f2ba40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:35.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.005+0000 7f2bbe20a640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ba40a3240 0x7f2ba40a5700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8102c80 tx=0x7f2bb400a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:35.008 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.008+0000 7f2baf7fe640 1 -- 172.21.15.136:0/1270662805 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bb009c0f0 con 0x7f2bb81037e0 2024-09-16T02:00:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:34 smithi172 ceph-mon[35340]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T02:00:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:34 smithi172 ceph-mon[35340]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T02:00:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:34 smithi172 ceph-mon[35340]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T02:00:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:34 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:34 smithi172 ceph-mon[35340]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T02:00:35.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:35.193+0000 7f2bc046e640 1 -- 172.21.15.136:0/1270662805 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2bb8062630 con 0x7f2ba40a3240 2024-09-16T02:00:35.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T02:00:35.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T02:00:35.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:35.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:35.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi103 2024-09-16T02:00:35.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:35.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:35.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:35.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:35.877 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:35.877 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-16T02:00:35.877 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:35.877 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:35.877 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:35.877 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:35.877 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:35.877 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:00:35.877 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:00:35.877 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:35 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:36.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T02:00:36.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T02:00:36.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:36.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:36.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi103 2024-09-16T02:00:36.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:36.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:00:36.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:35 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T02:00:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T02:00:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi103 2024-09-16T02:00:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:36.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:36.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:36.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:36.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:36.048 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-16T02:00:36.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:36.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:36.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:36.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:36.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:00:36.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:00:36.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:00:36.049 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:35 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:36.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:36 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T02:00:36.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:36 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T02:00:36.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:36 smithi136 ceph-mon[35323]: osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] boot 2024-09-16T02:00:36.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:36 smithi136 ceph-mon[35323]: osdmap e41: 7 total, 7 up, 7 in 2024-09-16T02:00:36.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:36 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:36.749 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:36 smithi136 ceph-mon[35323]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:36.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:36 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T02:00:36.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:36 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T02:00:36.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:36 smithi103 ceph-mon[30398]: osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] boot 2024-09-16T02:00:36.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:36 smithi103 ceph-mon[30398]: osdmap e41: 7 total, 7 up, 7 in 2024-09-16T02:00:36.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:36 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:36.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:36 smithi103 ceph-mon[30398]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:37.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:36 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T02:00:37.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:36 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T02:00:37.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:36 smithi172 ceph-mon[35340]: osd.6 [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] boot 2024-09-16T02:00:37.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:36 smithi172 ceph-mon[35340]: osdmap e41: 7 total, 7 up, 7 in 2024-09-16T02:00:37.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:36 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T02:00:37.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:36 smithi172 ceph-mon[35340]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:37.985 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:37 smithi136 ceph-mon[35323]: osdmap e42: 7 total, 7 up, 7 in 2024-09-16T02:00:37.985 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:37 smithi136 ceph-mon[35323]: from='client.? 172.21.15.136:0/161992267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c4b067-a2b3-404e-867e-8c8e51ecd4b2"}]: dispatch 2024-09-16T02:00:37.985 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:37 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c4b067-a2b3-404e-867e-8c8e51ecd4b2"}]: dispatch 2024-09-16T02:00:37.985 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:37 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c4b067-a2b3-404e-867e-8c8e51ecd4b2"}]': finished 2024-09-16T02:00:37.985 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:37 smithi136 ceph-mon[35323]: osdmap e43: 8 total, 7 up, 8 in 2024-09-16T02:00:37.985 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:37 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:00:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:37 smithi103 ceph-mon[30398]: osdmap e42: 7 total, 7 up, 7 in 2024-09-16T02:00:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:37 smithi103 ceph-mon[30398]: from='client.? 172.21.15.136:0/161992267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c4b067-a2b3-404e-867e-8c8e51ecd4b2"}]: dispatch 2024-09-16T02:00:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:37 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c4b067-a2b3-404e-867e-8c8e51ecd4b2"}]: dispatch 2024-09-16T02:00:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:37 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c4b067-a2b3-404e-867e-8c8e51ecd4b2"}]': finished 2024-09-16T02:00:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:37 smithi103 ceph-mon[30398]: osdmap e43: 8 total, 7 up, 8 in 2024-09-16T02:00:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:37 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:00:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:37 smithi172 ceph-mon[35340]: osdmap e42: 7 total, 7 up, 7 in 2024-09-16T02:00:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:37 smithi172 ceph-mon[35340]: from='client.? 172.21.15.136:0/161992267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c4b067-a2b3-404e-867e-8c8e51ecd4b2"}]: dispatch 2024-09-16T02:00:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:37 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c4b067-a2b3-404e-867e-8c8e51ecd4b2"}]: dispatch 2024-09-16T02:00:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:37 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c4b067-a2b3-404e-867e-8c8e51ecd4b2"}]': finished 2024-09-16T02:00:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:37 smithi172 ceph-mon[35340]: osdmap e43: 8 total, 7 up, 8 in 2024-09-16T02:00:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:37 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:00:39.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:38 smithi103 ceph-mon[30398]: from='client.? 172.21.15.136:0/491013321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:00:39.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:38 smithi103 ceph-mon[30398]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:38 smithi136 ceph-mon[35323]: from='client.? 172.21.15.136:0/491013321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:00:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:38 smithi136 ceph-mon[35323]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:39.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:38 smithi172 ceph-mon[35340]: from='client.? 172.21.15.136:0/491013321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:00:39.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:38 smithi172 ceph-mon[35340]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:41.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:41 smithi103 ceph-mon[30398]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:41.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:41 smithi136 ceph-mon[35323]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:41.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:41 smithi172 ceph-mon[35340]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:43.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:43 smithi103 ceph-mon[30398]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:43.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:43 smithi136 ceph-mon[35323]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:43.794 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:43 smithi172 ceph-mon[35340]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:45.716 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:45 smithi136 ceph-mon[35323]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:45.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:45 smithi103 ceph-mon[30398]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:45.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:45 smithi172 ceph-mon[35340]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:46.737 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:46 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T02:00:46.737 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:46 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:46.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:46 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T02:00:46.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:46 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:46.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:46 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T02:00:46.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:46 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:47.613 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:47 smithi136 ceph-mon[35323]: Deploying daemon osd.7 on smithi136 2024-09-16T02:00:47.613 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:47 smithi136 ceph-mon[35323]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:47 smithi103 ceph-mon[30398]: Deploying daemon osd.7 on smithi136 2024-09-16T02:00:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:47 smithi103 ceph-mon[30398]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:47 smithi172 ceph-mon[35340]: Deploying daemon osd.7 on smithi136 2024-09-16T02:00:47.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:47 smithi172 ceph-mon[35340]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:49.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:49 smithi103 ceph-mon[30398]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:49 smithi136 ceph-mon[35323]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:49.794 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:49 smithi172 ceph-mon[35340]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:51.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:51 smithi103 ceph-mon[30398]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:51.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:51 smithi136 ceph-mon[35323]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:51.794 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:51 smithi172 ceph-mon[35340]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:52.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:52 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:52.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:52 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:52.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:52 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:52.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:52 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:52.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:52 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:52.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:52 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:52.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:52 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:52.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:52 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:52.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:52 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:53.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:53 smithi103 ceph-mon[30398]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:53 smithi136 ceph-mon[35323]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:53 smithi172 ceph-mon[35340]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:55.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.033+0000 7f2baf7fe640 1 -- 172.21.15.136:0/1270662805 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2bb8062630 con 0x7f2ba40a3240 2024-09-16T02:00:55.037 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 7 on host 'smithi136' 2024-09-16T02:00:55.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 -- 172.21.15.136:0/1270662805 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ba40a3240 msgr2=0x7f2ba40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:55.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ba40a3240 0x7f2ba40a5700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8102c80 tx=0x7f2bb400a5e0 comp rx=0 tx=0).stop 2024-09-16T02:00:55.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 -- 172.21.15.136:0/1270662805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb81037e0 msgr2=0x7f2bb8074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:55.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb81037e0 0x7f2bb8074b20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000b970 tx=0x7f2bb000be30 comp rx=0 tx=0).stop 2024-09-16T02:00:55.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 -- 172.21.15.136:0/1270662805 shutdown_connections 2024-09-16T02:00:55.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ba40a3240 0x7f2ba40a5700 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:55.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb81037e0 0x7f2bb8074b20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:55.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bb8102e20 0x7f2bb80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:55.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 --2- 172.21.15.136:0/1270662805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bb8101c20 0x7f2bb8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:55.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 -- 172.21.15.136:0/1270662805 >> 172.21.15.136:0/1270662805 conn(0x7f2bb80fd3b0 msgr2=0x7f2bb81061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:55.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 -- 172.21.15.136:0/1270662805 shutdown_connections 2024-09-16T02:00:55.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:00:55.036+0000 7f2bc046e640 1 -- 172.21.15.136:0/1270662805 wait complete. 2024-09-16T02:00:55.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:55.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:55.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:55 smithi136 ceph-mon[35323]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:55.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:55.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:55 smithi103 ceph-mon[30398]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:55.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:55.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:55 smithi172 ceph-mon[35340]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:55.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:55.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:55.592 DEBUG:teuthology.orchestra.run.smithi136:osd.7> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T02:00:55.594 INFO:tasks.cephadm:Deploying osd.8 on smithi172 with /dev/vg_nvme/lv_2... 2024-09-16T02:00:55.595 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T02:00:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:00:57.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:57 smithi172 ceph-mon[35340]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:57.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:57 smithi103 ceph-mon[30398]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:57.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:57 smithi136 ceph-mon[35323]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:58.036 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:00:57 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:00:57.779+0000 7f7b459b6640 -1 osd.7 0 log_to_monitors true 2024-09-16T02:00:58.485 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T02:00:58.519 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_2 2024-09-16T02:00:58.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='osd.7 [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T02:00:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T02:00:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:58.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:58 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.769 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='osd.7 [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T02:00:58.769 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T02:00:58.769 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.769 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.769 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:58.770 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:58.770 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:58.770 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.770 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:58.770 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:58.770 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:58 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='osd.7 [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T02:00:58.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T02:00:58.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:58.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:58.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:00:58.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:58.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:58.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:58 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:00:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:00:59.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.194+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1818331618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d50104f90 msgr2=0x7f0d50107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:59.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.194+0000 7f0d57f0f640 1 --2- 172.21.15.172:0/1818331618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d50104f90 0x7f0d50107380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009900 tx=0x7f0d4c02a410 comp rx=0 tx=0).stop 2024-09-16T02:00:59.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.194+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1818331618 shutdown_connections 2024-09-16T02:00:59.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.194+0000 7f0d57f0f640 1 --2- 172.21.15.172:0/1818331618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d501078c0 0x7f0d50109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.194+0000 7f0d57f0f640 1 --2- 172.21.15.172:0/1818331618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d50104f90 0x7f0d50107380 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.194+0000 7f0d57f0f640 1 --2- 172.21.15.172:0/1818331618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d50102660 0x7f0d50104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.194+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1818331618 >> 172.21.15.172:0/1818331618 conn(0x7f0d500fc280 msgr2=0x7f0d500fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:59.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.194+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1818331618 shutdown_connections 2024-09-16T02:00:59.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.194+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1818331618 wait complete. 2024-09-16T02:00:59.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d57f0f640 1 Processor -- start 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d57f0f640 1 -- start start 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d57f0f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d50102660 0x7f0d50197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d57f0f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d50104f90 0x7f0d501984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d57f0f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d501078c0 0x7f0d5019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d57f0f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010d8c0 con 0x7f0d50102660 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d57f0f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010d740 con 0x7f0d50104f90 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d57f0f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010da40 con 0x7f0d501078c0 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d564ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d501078c0 0x7f0d5019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d564ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d501078c0 0x7f0d5019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46132/0 (socket says 172.21.15.172:46132) 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d564ac640 1 -- 172.21.15.172:0/1690217436 learned_addr learned my addr 172.21.15.172:0/1690217436 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d55cab640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d50102660 0x7f0d50197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d554aa640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d50104f90 0x7f0d501984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:59.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d564ac640 1 -- 172.21.15.172:0/1690217436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d50104f90 msgr2=0x7f0d501984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:59.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d564ac640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d50104f90 0x7f0d501984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d564ac640 1 -- 172.21.15.172:0/1690217436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d50102660 msgr2=0x7f0d50197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:59.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d564ac640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d50102660 0x7f0d50197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:59.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.195+0000 7f0d564ac640 1 -- 172.21.15.172:0/1690217436 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5019fbe0 con 0x7f0d501078c0 2024-09-16T02:00:59.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.196+0000 7f0d564ac640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d501078c0 0x7f0d5019f570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800eaa0 tx=0x7f0d4800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:59.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.196+0000 7f0d46ffd640 1 -- 172.21.15.172:0/1690217436 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48004d20 con 0x7f0d501078c0 2024-09-16T02:00:59.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.196+0000 7f0d46ffd640 1 -- 172.21.15.172:0/1690217436 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d48005020 con 0x7f0d501078c0 2024-09-16T02:00:59.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.196+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1690217436 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5019fed0 con 0x7f0d501078c0 2024-09-16T02:00:59.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.196+0000 7f0d46ffd640 1 -- 172.21.15.172:0/1690217436 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d480116d0 con 0x7f0d501078c0 2024-09-16T02:00:59.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.196+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1690217436 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a03b0 con 0x7f0d501078c0 2024-09-16T02:00:59.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.197+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1690217436 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d500fdd70 con 0x7f0d501078c0 2024-09-16T02:00:59.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.198+0000 7f0d46ffd640 1 -- 172.21.15.172:0/1690217436 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0d480051d0 con 0x7f0d501078c0 2024-09-16T02:00:59.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.199+0000 7f0d46ffd640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d3c0a3270 0x7f0d3c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:59.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.199+0000 7f0d46ffd640 1 -- 172.21.15.172:0/1690217436 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5101+0+0 (secure 0 0 0) 0x7f0d480cede0 con 0x7f0d501078c0 2024-09-16T02:00:59.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.199+0000 7f0d55cab640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d3c0a3270 0x7f0d3c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:59.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.200+0000 7f0d55cab640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d3c0a3270 0x7f0d3c0a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009990 tx=0x7f0d40009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:59.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.203+0000 7f0d46ffd640 1 -- 172.21.15.172:0/1690217436 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d48016030 con 0x7f0d501078c0 2024-09-16T02:00:59.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:00:59.386+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1690217436 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0d50062630 con 0x7f0d3c0a3270 2024-09-16T02:00:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi136 2024-09-16T02:00:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi136 to 5696M 2024-09-16T02:00:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T02:00:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: from='osd.7 [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:00:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: osdmap e44: 8 total, 7 up, 8 in 2024-09-16T02:00:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:00:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:00:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:00:59.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:00:59.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:59.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T02:00:59.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: osdmap e45: 8 total, 7 up, 8 in 2024-09-16T02:00:59.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:00:59 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:00:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi136 2024-09-16T02:00:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi136 to 5696M 2024-09-16T02:00:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T02:00:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: from='osd.7 [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:00:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: osdmap e44: 8 total, 7 up, 8 in 2024-09-16T02:00:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:00:59.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:00:59.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:00:59.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:00:59.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:59.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T02:00:59.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: osdmap e45: 8 total, 7 up, 8 in 2024-09-16T02:00:59.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:00:59 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:00:59.779 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi136 2024-09-16T02:00:59.779 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi136 to 5696M 2024-09-16T02:00:59.779 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: from='osd.7 [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: osdmap e44: 8 total, 7 up, 8 in 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: osdmap e45: 8 total, 7 up, 8 in 2024-09-16T02:00:59.780 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:00:59 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:01:00.484 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:01:00 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:01:00.133+0000 7f7b41116640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T02:01:00.484 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:01:00 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:01:00.146+0000 7f7b39748640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T02:01:00.750 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:00 smithi172 ceph-mon[35340]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:01:00.750 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:01:00.750 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:00 smithi172 ceph-mon[35340]: from='osd.7 ' entity='osd.7' 2024-09-16T02:01:00.750 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:00 smithi172 ceph-mon[35340]: osd.7 [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] boot 2024-09-16T02:01:00.750 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:00 smithi172 ceph-mon[35340]: osdmap e46: 8 total, 8 up, 8 in 2024-09-16T02:01:00.750 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:01:00.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:00 smithi103 ceph-mon[30398]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:01:00.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:01:00.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:00 smithi103 ceph-mon[30398]: from='osd.7 ' entity='osd.7' 2024-09-16T02:01:00.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:00 smithi103 ceph-mon[30398]: osd.7 [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] boot 2024-09-16T02:01:00.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:00 smithi103 ceph-mon[30398]: osdmap e46: 8 total, 8 up, 8 in 2024-09-16T02:01:00.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:01:00.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:00 smithi136 ceph-mon[35323]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:01:00.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:01:00.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:00 smithi136 ceph-mon[35323]: from='osd.7 ' entity='osd.7' 2024-09-16T02:01:00.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:00 smithi136 ceph-mon[35323]: osd.7 [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] boot 2024-09-16T02:01:00.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:00 smithi136 ceph-mon[35323]: osdmap e46: 8 total, 8 up, 8 in 2024-09-16T02:01:00.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T02:01:01.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:01 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T02:01:01.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:01 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T02:01:01.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:01 smithi103 ceph-mon[30398]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:01:01.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:01 smithi103 ceph-mon[30398]: osdmap e47: 8 total, 8 up, 8 in 2024-09-16T02:01:01.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:01 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T02:01:01.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:01 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T02:01:01.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:01 smithi136 ceph-mon[35323]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:01:01.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:01 smithi136 ceph-mon[35323]: osdmap e47: 8 total, 8 up, 8 in 2024-09-16T02:01:01.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:01 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T02:01:01.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:01 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T02:01:01.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:01 smithi172 ceph-mon[35340]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T02:01:01.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:01 smithi172 ceph-mon[35340]: osdmap e47: 8 total, 8 up, 8 in 2024-09-16T02:01:02.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:02 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/976211845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b835bf50-420f-42f9-8ca5-c3f1527be672"}]: dispatch 2024-09-16T02:01:02.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:02 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b835bf50-420f-42f9-8ca5-c3f1527be672"}]: dispatch 2024-09-16T02:01:02.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:02 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b835bf50-420f-42f9-8ca5-c3f1527be672"}]': finished 2024-09-16T02:01:02.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:02 smithi103 ceph-mon[30398]: osdmap e48: 9 total, 8 up, 9 in 2024-09-16T02:01:02.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:02 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:02.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:02 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/1967747539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:01:02.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:02 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/976211845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b835bf50-420f-42f9-8ca5-c3f1527be672"}]: dispatch 2024-09-16T02:01:02.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:02 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b835bf50-420f-42f9-8ca5-c3f1527be672"}]: dispatch 2024-09-16T02:01:02.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:02 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b835bf50-420f-42f9-8ca5-c3f1527be672"}]': finished 2024-09-16T02:01:02.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:02 smithi136 ceph-mon[35323]: osdmap e48: 9 total, 8 up, 9 in 2024-09-16T02:01:02.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:02 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:02.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:02 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/1967747539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:01:02.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:02 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/976211845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b835bf50-420f-42f9-8ca5-c3f1527be672"}]: dispatch 2024-09-16T02:01:02.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:02 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b835bf50-420f-42f9-8ca5-c3f1527be672"}]: dispatch 2024-09-16T02:01:02.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:02 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b835bf50-420f-42f9-8ca5-c3f1527be672"}]': finished 2024-09-16T02:01:02.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:02 smithi172 ceph-mon[35340]: osdmap e48: 9 total, 8 up, 9 in 2024-09-16T02:01:02.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:02 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:02.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:02 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/1967747539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:01:03.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:03 smithi103 ceph-mon[30398]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:03.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:03 smithi103 ceph-mon[30398]: osdmap e49: 9 total, 8 up, 9 in 2024-09-16T02:01:03.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:03 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:03.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:03 smithi136 ceph-mon[35323]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:03.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:03 smithi136 ceph-mon[35323]: osdmap e49: 9 total, 8 up, 9 in 2024-09-16T02:01:03.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:03 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:03.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:03 smithi172 ceph-mon[35340]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:03.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:03 smithi172 ceph-mon[35340]: osdmap e49: 9 total, 8 up, 9 in 2024-09-16T02:01:03.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:03 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:05.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:04 smithi103 ceph-mon[30398]: osdmap e50: 9 total, 8 up, 9 in 2024-09-16T02:01:05.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:04 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:05.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:04 smithi103 ceph-mon[30398]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:05.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:04 smithi136 ceph-mon[35323]: osdmap e50: 9 total, 8 up, 9 in 2024-09-16T02:01:05.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:04 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:05.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:04 smithi136 ceph-mon[35323]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:05.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:04 smithi172 ceph-mon[35340]: osdmap e50: 9 total, 8 up, 9 in 2024-09-16T02:01:05.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:04 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:05.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:04 smithi172 ceph-mon[35340]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:07.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:07 smithi103 ceph-mon[30398]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:07.767 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:07 smithi172 ceph-mon[35340]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:07.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:07 smithi136 ceph-mon[35323]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:09.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:09 smithi103 ceph-mon[30398]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:09.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:09 smithi136 ceph-mon[35323]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:09.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:09 smithi172 ceph-mon[35340]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:11.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:11 smithi172 ceph-mon[35340]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:11.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T02:01:11.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:11.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:11 smithi172 ceph-mon[35340]: Deploying daemon osd.8 on smithi172 2024-09-16T02:01:11.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:11 smithi103 ceph-mon[30398]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:11.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T02:01:11.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:11.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:11 smithi103 ceph-mon[30398]: Deploying daemon osd.8 on smithi172 2024-09-16T02:01:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:11 smithi136 ceph-mon[35323]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T02:01:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:11.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:11 smithi136 ceph-mon[35323]: Deploying daemon osd.8 on smithi172 2024-09-16T02:01:13.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:13 smithi172 ceph-mon[35340]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:13.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:13 smithi103 ceph-mon[30398]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:13.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:13 smithi136 ceph-mon[35323]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:15.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:15 smithi103 ceph-mon[30398]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:15.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:15 smithi136 ceph-mon[35323]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:15.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:15 smithi172 ceph-mon[35340]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:16.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:16.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:16.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:16.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:16.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:16.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:16.794 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:16.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:16.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:17.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:17 smithi103 ceph-mon[30398]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:17.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:17 smithi136 ceph-mon[35323]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:17.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:17 smithi172 ceph-mon[35340]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:19 smithi103 ceph-mon[30398]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:19.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:19 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:19 smithi136 ceph-mon[35323]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:19.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:19 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:19 smithi172 ceph-mon[35340]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:19 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:20.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.227+0000 7f0d46ffd640 1 -- 172.21.15.172:0/1690217436 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0d50062630 con 0x7f0d3c0a3270 2024-09-16T02:01:20.229 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 8 on host 'smithi172' 2024-09-16T02:01:20.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.229+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1690217436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d3c0a3270 msgr2=0x7f0d3c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.229+0000 7f0d57f0f640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d3c0a3270 0x7f0d3c0a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009990 tx=0x7f0d40009110 comp rx=0 tx=0).stop 2024-09-16T02:01:20.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.229+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1690217436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d501078c0 msgr2=0x7f0d5019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:20.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.229+0000 7f0d57f0f640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d501078c0 0x7f0d5019f570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800eaa0 tx=0x7f0d4800eea0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.229+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1690217436 shutdown_connections 2024-09-16T02:01:20.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.229+0000 7f0d57f0f640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0d3c0a3270 0x7f0d3c0a5730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.229+0000 7f0d57f0f640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d501078c0 0x7f0d5019f570 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.229+0000 7f0d57f0f640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d50104f90 0x7f0d501984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.229+0000 7f0d57f0f640 1 --2- 172.21.15.172:0/1690217436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d50102660 0x7f0d50197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:20.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.229+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1690217436 >> 172.21.15.172:0/1690217436 conn(0x7f0d500fc280 msgr2=0x7f0d50103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:20.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.230+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1690217436 shutdown_connections 2024-09-16T02:01:20.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:01:20.230+0000 7f0d57f0f640 1 -- 172.21.15.172:0/1690217436 wait complete. 2024-09-16T02:01:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:20.912 DEBUG:teuthology.orchestra.run.smithi172:osd.8> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.8.service 2024-09-16T02:01:20.914 INFO:tasks.cephadm:Deploying osd.9 on smithi103 with /dev/vg_nvme/lv_1... 2024-09-16T02:01:20.914 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T02:01:20.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:20.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:20.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:21.291 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:21.687 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:21 smithi103 ceph-mon[30398]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:21 smithi136 ceph-mon[35323]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:21.782 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:21 smithi172 ceph-mon[35340]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:22.706 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:22 smithi103 ceph-mon[30398]: from='osd.8 [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T02:01:22.707 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:22 smithi103 ceph-mon[30398]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T02:01:22.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:22 smithi136 ceph-mon[35323]: from='osd.8 [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T02:01:22.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:22 smithi136 ceph-mon[35323]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T02:01:22.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:22 smithi172 ceph-mon[35340]: from='osd.8 [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T02:01:22.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:22 smithi172 ceph-mon[35340]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T02:01:22.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:01:22 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:01:22.501+0000 7f73b7ab2640 -1 osd.8 0 log_to_monitors true 2024-09-16T02:01:23.606 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:23 smithi172 ceph-mon[35340]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:23.606 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:23 smithi172 ceph-mon[35340]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T02:01:23.606 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:23 smithi172 ceph-mon[35340]: from='osd.8 [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:01:23.606 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:23 smithi172 ceph-mon[35340]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T02:01:23.606 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:23.606 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:23 smithi172 ceph-mon[35340]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:01:23.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:23 smithi103 ceph-mon[30398]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:23.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:23 smithi103 ceph-mon[30398]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T02:01:23.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:23 smithi103 ceph-mon[30398]: from='osd.8 [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:01:23.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:23 smithi103 ceph-mon[30398]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T02:01:23.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:23.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:23 smithi103 ceph-mon[30398]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:01:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:23 smithi136 ceph-mon[35323]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:23 smithi136 ceph-mon[35323]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T02:01:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:23 smithi136 ceph-mon[35323]: from='osd.8 [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:01:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:23 smithi136 ceph-mon[35323]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T02:01:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:23 smithi136 ceph-mon[35323]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:01:24.197 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:01:24.229 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-09-16T02:01:24.610 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:01:24.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:01:24.363+0000 7f73b3212640 -1 osd.8 0 waiting for initial osdmap 2024-09-16T02:01:24.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:01:24.377+0000 7f73ab844640 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T02:01:24.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:24.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:24.797 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:24 smithi172 ceph-mon[35340]: from='osd.8 ' entity='osd.8' 2024-09-16T02:01:24.894 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:24.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:24.896 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:24 smithi103 ceph-mon[30398]: from='osd.8 ' entity='osd.8' 2024-09-16T02:01:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T02:01:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T02:01:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:25.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:25.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:25.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:25.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:25.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:25.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:25.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:25.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:25.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:24 smithi136 ceph-mon[35323]: from='osd.8 ' entity='osd.8' 2024-09-16T02:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.156+0000 7f8bff800640 1 -- 172.21.15.103:0/1130917841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf8101c00 msgr2=0x7f8bf8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.156+0000 7f8bff800640 1 --2- 172.21.15.103:0/1130917841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf8101c00 0x7f8bf8102000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009900 tx=0x7f8bec02a3f0 comp rx=0 tx=0).stop 2024-09-16T02:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.156+0000 7f8bff800640 1 -- 172.21.15.103:0/1130917841 shutdown_connections 2024-09-16T02:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.156+0000 7f8bff800640 1 --2- 172.21.15.103:0/1130917841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf81037c0 0x7f8bf810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.156+0000 7f8bff800640 1 --2- 172.21.15.103:0/1130917841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf8102e00 0x7f8bf8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.156+0000 7f8bff800640 1 --2- 172.21.15.103:0/1130917841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf8101c00 0x7f8bf8102000 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.156+0000 7f8bff800640 1 -- 172.21.15.103:0/1130917841 >> 172.21.15.103:0/1130917841 conn(0x7f8bf80fd390 msgr2=0x7f8bf80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.156+0000 7f8bff800640 1 -- 172.21.15.103:0/1130917841 shutdown_connections 2024-09-16T02:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.156+0000 7f8bff800640 1 -- 172.21.15.103:0/1130917841 wait complete. 2024-09-16T02:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.157+0000 7f8bff800640 1 Processor -- start 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.157+0000 7f8bff800640 1 -- start start 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.157+0000 7f8bff800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf8101c00 0x7f8bf8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.157+0000 7f8bff800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf8102e00 0x7f8bf80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bff800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf81037c0 0x7f8bf8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bff800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf810d8e0 con 0x7f8bf8102e00 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bff800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf810d760 con 0x7f8bf8101c00 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bff800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf810da60 con 0x7f8bf81037c0 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bf7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf8102e00 0x7f8bf80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bfd800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf81037c0 0x7f8bf8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bf7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf8102e00 0x7f8bf80745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35496/0 (socket says 172.21.15.103:35496) 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bf7e00640 1 -- 172.21.15.103:0/2772050332 learned_addr learned my addr 172.21.15.103:0/2772050332 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bfce00640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf8101c00 0x7f8bf8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bf7e00640 1 -- 172.21.15.103:0/2772050332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf81037c0 msgr2=0x7f8bf8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bf7e00640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf81037c0 0x7f8bf8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bf7e00640 1 -- 172.21.15.103:0/2772050332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf8101c00 msgr2=0x7f8bf8075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bf7e00640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf8101c00 0x7f8bf8075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bf7e00640 1 -- 172.21.15.103:0/2772050332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf80753e0 con 0x7f8bf8102e00 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.158+0000 7f8bfd800640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf81037c0 0x7f8bf8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.159+0000 7f8bf7e00640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf8102e00 0x7f8bf80745e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8be400ea60 tx=0x7f8be400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.159+0000 7f8bf5600640 1 -- 172.21.15.103:0/2772050332 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be400cde0 con 0x7f8bf8102e00 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.159+0000 7f8bff800640 1 -- 172.21.15.103:0/2772050332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf81a2060 con 0x7f8bf8102e00 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.159+0000 7f8bf5600640 1 -- 172.21.15.103:0/2772050332 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be40035f0 con 0x7f8bf8102e00 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.159+0000 7f8bf5600640 1 -- 172.21.15.103:0/2772050332 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4010670 con 0x7f8bf8102e00 2024-09-16T02:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.159+0000 7f8bff800640 1 -- 172.21.15.103:0/2772050332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf81a25c0 con 0x7f8bf8102e00 2024-09-16T02:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.161+0000 7f8bff800640 1 -- 172.21.15.103:0/2772050332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf8108fb0 con 0x7f8bf8102e00 2024-09-16T02:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.163+0000 7f8bf5600640 1 -- 172.21.15.103:0/2772050332 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8be40037a0 con 0x7f8bf8102e00 2024-09-16T02:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.165+0000 7f8bf5600640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8be00a32c0 0x7f8be00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.165+0000 7f8bf5600640 1 -- 172.21.15.103:0/2772050332 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5862+0+0 (secure 0 0 0) 0x7f8be40cf250 con 0x7f8bf8102e00 2024-09-16T02:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.165+0000 7f8bfce00640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8be00a32c0 0x7f8be00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.166+0000 7f8bfce00640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8be00a32c0 0x7f8be00a5780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8bec02a9e0 tx=0x7f8bec0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.169+0000 7f8bf5600640 1 -- 172.21.15.103:0/2772050332 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8be409c860 con 0x7f8bf8102e00 2024-09-16T02:01:25.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:25.353+0000 7f8bff800640 1 -- 172.21.15.103:0/2772050332 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8bf8062630 con 0x7f8be00a32c0 2024-09-16T02:01:25.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi172 2024-09-16T02:01:25.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-16T02:01:25.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T02:01:25.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T02:01:25.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:25.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:25.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: osd.8 [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] boot 2024-09-16T02:01:25.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: osdmap e53: 9 total, 9 up, 9 in 2024-09-16T02:01:25.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:25.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:01:25.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:01:25.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:25.946 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi172 2024-09-16T02:01:25.947 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-16T02:01:25.947 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T02:01:25.947 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T02:01:25.947 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:25.947 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:25.947 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: osd.8 [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] boot 2024-09-16T02:01:25.947 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: osdmap e53: 9 total, 9 up, 9 in 2024-09-16T02:01:25.947 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:25.948 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:01:25.948 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:01:25.948 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi172 2024-09-16T02:01:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-16T02:01:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T02:01:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T02:01:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T02:01:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: osd.8 [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] boot 2024-09-16T02:01:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: osdmap e53: 9 total, 9 up, 9 in 2024-09-16T02:01:26.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T02:01:26.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:01:26.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:01:26.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:26 smithi172 ceph-mon[35340]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:01:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:26 smithi172 ceph-mon[35340]: osdmap e54: 9 total, 9 up, 9 in 2024-09-16T02:01:26.956 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:26 smithi103 ceph-mon[30398]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:01:26.956 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:26 smithi103 ceph-mon[30398]: osdmap e54: 9 total, 9 up, 9 in 2024-09-16T02:01:27.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:26 smithi136 ceph-mon[35323]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:01:27.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:26 smithi136 ceph-mon[35323]: osdmap e54: 9 total, 9 up, 9 in 2024-09-16T02:01:27.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:27 smithi103 ceph-mon[30398]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:27.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:27 smithi103 ceph-mon[30398]: osdmap e55: 9 total, 9 up, 9 in 2024-09-16T02:01:27.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:27 smithi172 ceph-mon[35340]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:27.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:27 smithi172 ceph-mon[35340]: osdmap e55: 9 total, 9 up, 9 in 2024-09-16T02:01:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:27 smithi136 ceph-mon[35323]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:27 smithi136 ceph-mon[35323]: osdmap e55: 9 total, 9 up, 9 in 2024-09-16T02:01:28.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:28 smithi103 ceph-mon[30398]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T02:01:28.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:28 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2619305015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46691205-fd3d-4f55-92bf-eae5bc09f6d7"}]: dispatch 2024-09-16T02:01:28.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:28 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46691205-fd3d-4f55-92bf-eae5bc09f6d7"}]: dispatch 2024-09-16T02:01:28.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:28 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46691205-fd3d-4f55-92bf-eae5bc09f6d7"}]': finished 2024-09-16T02:01:28.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:28 smithi103 ceph-mon[30398]: osdmap e57: 10 total, 9 up, 10 in 2024-09-16T02:01:28.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:28 smithi136 ceph-mon[35323]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T02:01:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:28 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2619305015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46691205-fd3d-4f55-92bf-eae5bc09f6d7"}]: dispatch 2024-09-16T02:01:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:28 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46691205-fd3d-4f55-92bf-eae5bc09f6d7"}]: dispatch 2024-09-16T02:01:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:28 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46691205-fd3d-4f55-92bf-eae5bc09f6d7"}]': finished 2024-09-16T02:01:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:28 smithi136 ceph-mon[35323]: osdmap e57: 10 total, 9 up, 10 in 2024-09-16T02:01:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:28 smithi172 ceph-mon[35340]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T02:01:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:28 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2619305015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46691205-fd3d-4f55-92bf-eae5bc09f6d7"}]: dispatch 2024-09-16T02:01:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:28 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46691205-fd3d-4f55-92bf-eae5bc09f6d7"}]: dispatch 2024-09-16T02:01:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:28 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46691205-fd3d-4f55-92bf-eae5bc09f6d7"}]': finished 2024-09-16T02:01:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:28 smithi172 ceph-mon[35340]: osdmap e57: 10 total, 9 up, 10 in 2024-09-16T02:01:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:30.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:29 smithi103 ceph-mon[30398]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:30.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:29 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3617069822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:01:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:29 smithi136 ceph-mon[35323]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:29 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3617069822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:01:30.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:29 smithi172 ceph-mon[35340]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:30.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:29 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3617069822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:01:32.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:31 smithi103 ceph-mon[30398]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:31 smithi136 ceph-mon[35323]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:31 smithi172 ceph-mon[35340]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:33.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:33 smithi103 ceph-mon[30398]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T02:01:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:33 smithi136 ceph-mon[35323]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T02:01:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:33 smithi172 ceph-mon[35340]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T02:01:35.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:35 smithi103 ceph-mon[30398]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-16T02:01:36.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:35 smithi136 ceph-mon[35323]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-16T02:01:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:35 smithi172 ceph-mon[35340]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-16T02:01:37.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:37 smithi103 ceph-mon[30398]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-16T02:01:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:37 smithi136 ceph-mon[35323]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-16T02:01:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:37 smithi172 ceph-mon[35340]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-16T02:01:38.864 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:38 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T02:01:38.865 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:38 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:38.867 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:38 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T02:01:38.867 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:38 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:38 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T02:01:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:38 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:39.924 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:39 smithi136 ceph-mon[35323]: Deploying daemon osd.9 on smithi103 2024-09-16T02:01:39.924 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:39 smithi136 ceph-mon[35323]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-16T02:01:40.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:39 smithi103 ceph-mon[30398]: Deploying daemon osd.9 on smithi103 2024-09-16T02:01:40.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:39 smithi103 ceph-mon[30398]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-16T02:01:40.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:39 smithi172 ceph-mon[35340]: Deploying daemon osd.9 on smithi103 2024-09-16T02:01:40.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:39 smithi172 ceph-mon[35340]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-16T02:01:41.726 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:41 smithi103 ceph-mon[30398]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T02:01:42.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:41 smithi136 ceph-mon[35323]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T02:01:42.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:41 smithi172 ceph-mon[35340]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T02:01:44.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:43 smithi103 ceph-mon[30398]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T02:01:44.022 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:43 smithi136 ceph-mon[35323]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T02:01:44.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:43 smithi172 ceph-mon[35340]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T02:01:45.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:45 smithi103 ceph-mon[30398]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:45.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:45 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:45.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:45 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:45.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:45 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:46.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:45 smithi136 ceph-mon[35323]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:46.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:45 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:46.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:45 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:46.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:45 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:46.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:45 smithi172 ceph-mon[35340]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:46.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:45 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:46.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:45 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:46.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:45 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:47.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:47 smithi103 ceph-mon[30398]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:48.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:47 smithi136 ceph-mon[35323]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:48.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:47 smithi172 ceph-mon[35340]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:49.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.275+0000 7f8bf5600640 1 -- 172.21.15.103:0/2772050332 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8bf8062630 con 0x7f8be00a32c0 2024-09-16T02:01:49.278 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 9 on host 'smithi103' 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 -- 172.21.15.103:0/2772050332 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8be00a32c0 msgr2=0x7f8be00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8be00a32c0 0x7f8be00a5780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8bec02a9e0 tx=0x7f8bec0315b0 comp rx=0 tx=0).stop 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 -- 172.21.15.103:0/2772050332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf8102e00 msgr2=0x7f8bf80745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf8102e00 0x7f8bf80745e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8be400ea60 tx=0x7f8be400ef20 comp rx=0 tx=0).stop 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 -- 172.21.15.103:0/2772050332 shutdown_connections 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8be00a32c0 0x7f8be00a5780 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf81037c0 0x7f8bf8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf8102e00 0x7f8bf80745e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 --2- 172.21.15.103:0/2772050332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf8101c00 0x7f8bf8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 -- 172.21.15.103:0/2772050332 >> 172.21.15.103:0/2772050332 conn(0x7f8bf80fd390 msgr2=0x7f8bf8106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 -- 172.21.15.103:0/2772050332 shutdown_connections 2024-09-16T02:01:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:01:49.278+0000 7f8bff800640 1 -- 172.21.15.103:0/2772050332 wait complete. 2024-09-16T02:01:49.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:49.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:49.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:49 smithi103 ceph-mon[30398]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:49.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:49.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.588 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.588 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:49.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:49.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:49 smithi172 ceph-mon[35340]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:49.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:49.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:49.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:49.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:49 smithi136 ceph-mon[35323]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:49.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:01:49.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:49.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:50.000 DEBUG:teuthology.orchestra.run.smithi103:osd.9> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.9.service 2024-09-16T02:01:50.004 INFO:tasks.cephadm:Deploying osd.10 on smithi136 with /dev/vg_nvme/lv_1... 2024-09-16T02:01:50.005 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T02:01:50.261 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:01:51.634 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:51 smithi136 ceph-mon[35323]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:51 smithi103 ceph-mon[30398]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:51.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:51 smithi172 ceph-mon[35340]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:52.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:01:51 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:01:51.793+0000 7f544c630640 -1 osd.9 0 log_to_monitors true 2024-09-16T02:01:52.654 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:52 smithi136 ceph-mon[35323]: from='osd.9 [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T02:01:52.654 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:52 smithi136 ceph-mon[35323]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T02:01:52.656 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T02:01:52.685 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_1 2024-09-16T02:01:52.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:52 smithi103 ceph-mon[30398]: from='osd.9 [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T02:01:52.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:52 smithi103 ceph-mon[30398]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T02:01:52.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:52 smithi172 ceph-mon[35340]: from='osd.9 [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T02:01:52.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:52 smithi172 ceph-mon[35340]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T02:01:52.936 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:01:53.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.340+0000 7f645cf8c640 1 -- 172.21.15.136:0/124521161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6458102690 msgr2=0x7f6458108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:53.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.340+0000 7f645cf8c640 1 --2- 172.21.15.136:0/124521161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6458102690 0x7f6458108f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6450009ca0 tx=0x7f645002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.340+0000 7f645cf8c640 1 -- 172.21.15.136:0/124521161 shutdown_connections 2024-09-16T02:01:53.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.340+0000 7f645cf8c640 1 --2- 172.21.15.136:0/124521161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6458102690 0x7f6458108f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.340+0000 7f645cf8c640 1 --2- 172.21.15.136:0/124521161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6458101cd0 0x7f6458102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.340+0000 7f645cf8c640 1 --2- 172.21.15.136:0/124521161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6458100ad0 0x7f6458100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.340+0000 7f645cf8c640 1 -- 172.21.15.136:0/124521161 >> 172.21.15.136:0/124521161 conn(0x7f64580fc280 msgr2=0x7f64580fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:01:53.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.340+0000 7f645cf8c640 1 -- 172.21.15.136:0/124521161 shutdown_connections 2024-09-16T02:01:53.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.340+0000 7f645cf8c640 1 -- 172.21.15.136:0/124521161 wait complete. 2024-09-16T02:01:53.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645cf8c640 1 Processor -- start 2024-09-16T02:01:53.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645cf8c640 1 -- start start 2024-09-16T02:01:53.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645cf8c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6458100ad0 0x7f645806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:53.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645cf8c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6458101cd0 0x7f645819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:53.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645cf8c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6458102690 0x7f645819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:53.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645cf8c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f645810c730 con 0x7f6458101cd0 2024-09-16T02:01:53.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645cf8c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f645810c5b0 con 0x7f6458100ad0 2024-09-16T02:01:53.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645cf8c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f645810c8b0 con 0x7f6458102690 2024-09-16T02:01:53.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6458100ad0 0x7f645806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:53.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6458100ad0 0x7f645806aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:50692/0 (socket says 172.21.15.136:50692) 2024-09-16T02:01:53.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.341+0000 7f645659c640 1 -- 172.21.15.136:0/3192416179 learned_addr learned my addr 172.21.15.136:0/3192416179 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-16T02:01:53.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f6455d9b640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6458101cd0 0x7f645819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:53.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f6456d9d640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6458102690 0x7f645819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:53.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f645659c640 1 -- 172.21.15.136:0/3192416179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6458102690 msgr2=0x7f645819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:53.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f645659c640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6458102690 0x7f645819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f645659c640 1 -- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6458101cd0 msgr2=0x7f645819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:01:53.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f645659c640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6458101cd0 0x7f645819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:01:53.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f645659c640 1 -- 172.21.15.136:0/3192416179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645819fa50 con 0x7f6458100ad0 2024-09-16T02:01:53.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f6455d9b640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6458101cd0 0x7f645819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:01:53.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f6456d9d640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6458102690 0x7f645819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:01:53.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f645659c640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6458100ad0 0x7f645806aad0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c970 tx=0x7f644c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:53.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f64477fe640 1 -- 172.21.15.136:0/3192416179 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c007c10 con 0x7f6458100ad0 2024-09-16T02:01:53.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f64477fe640 1 -- 172.21.15.136:0/3192416179 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644c007db0 con 0x7f6458100ad0 2024-09-16T02:01:53.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f645cf8c640 1 -- 172.21.15.136:0/3192416179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645819fce0 con 0x7f6458100ad0 2024-09-16T02:01:53.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f64477fe640 1 -- 172.21.15.136:0/3192416179 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c00f940 con 0x7f6458100ad0 2024-09-16T02:01:53.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.342+0000 7f645cf8c640 1 -- 172.21.15.136:0/3192416179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64581a0210 con 0x7f6458100ad0 2024-09-16T02:01:53.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.343+0000 7f645cf8c640 1 -- 172.21.15.136:0/3192416179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6458107e80 con 0x7f6458100ad0 2024-09-16T02:01:53.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.345+0000 7f64477fe640 1 -- 172.21.15.136:0/3192416179 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f644c00fb40 con 0x7f6458100ad0 2024-09-16T02:01:53.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.345+0000 7f64477fe640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f64400a32c0 0x7f64400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:01:53.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.345+0000 7f64477fe640 1 -- 172.21.15.136:0/3192416179 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5965+0+0 (secure 0 0 0) 0x7f644c0cf660 con 0x7f6458100ad0 2024-09-16T02:01:53.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.346+0000 7f6455d9b640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f64400a32c0 0x7f64400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:01:53.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.346+0000 7f6455d9b640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f64400a32c0 0x7f64400a5780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f64480058c0 tx=0x7f6448005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:01:53.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.349+0000 7f64477fe640 1 -- 172.21.15.136:0/3192416179 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f644c09cdd0 con 0x7f6458100ad0 2024-09-16T02:01:53.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:01:53.539+0000 7f645cf8c640 1 -- 172.21.15.136:0/3192416179 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6458062630 con 0x7f64400a32c0 2024-09-16T02:01:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T02:01:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T02:01:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='osd.9 [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T02:01:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T02:01:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:53.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:53.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T02:01:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T02:01:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='osd.9 [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T02:01:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T02:01:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:53.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:53.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T02:01:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='osd.9 [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:01:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:01:54.754 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:01:54.367+0000 7f5446400640 -1 osd.9 0 waiting for initial osdmap 2024-09-16T02:01:54.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:01:54.393+0000 7f543ce00640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T02:01:54.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi103 2024-09-16T02:01:54.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-16T02:01:54.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T02:01:54.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T02:01:54.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:54.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:54.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:01:54.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:01:54.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:01:54.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:54.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:54 smithi103 ceph-mon[30398]: from='osd.9 ' entity='osd.9' 2024-09-16T02:01:54.771 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi103 2024-09-16T02:01:54.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-16T02:01:54.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T02:01:54.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T02:01:54.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:54.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:54.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:01:54.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:01:54.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:01:54.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:54.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:54 smithi136 ceph-mon[35323]: from='osd.9 ' entity='osd.9' 2024-09-16T02:01:54.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi103 2024-09-16T02:01:54.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-16T02:01:54.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-16T02:01:54.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T02:01:54.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:54.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:54.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:01:54.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:01:54.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:01:54.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:01:54.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:54 smithi172 ceph-mon[35340]: from='osd.9 ' entity='osd.9' 2024-09-16T02:01:55.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:55 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T02:01:55.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:55 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T02:01:55.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:55 smithi103 ceph-mon[30398]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:55.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:55.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:55 smithi103 ceph-mon[30398]: osd.9 [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] boot 2024-09-16T02:01:55.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:55 smithi103 ceph-mon[30398]: osdmap e60: 10 total, 10 up, 10 in 2024-09-16T02:01:55.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:55 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:55.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:55 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T02:01:55.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:55 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T02:01:55.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:55 smithi136 ceph-mon[35323]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:55.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:55.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:55 smithi136 ceph-mon[35323]: osd.9 [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] boot 2024-09-16T02:01:55.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:55 smithi136 ceph-mon[35323]: osdmap e60: 10 total, 10 up, 10 in 2024-09-16T02:01:55.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:55 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:55 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T02:01:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:55 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T02:01:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:55 smithi172 ceph-mon[35340]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-16T02:01:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:55 smithi172 ceph-mon[35340]: osd.9 [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] boot 2024-09-16T02:01:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:55 smithi172 ceph-mon[35340]: osdmap e60: 10 total, 10 up, 10 in 2024-09-16T02:01:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:55 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T02:01:56.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:56 smithi103 ceph-mon[30398]: from='client.? 172.21.15.136:0/4114173798' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f31571-16de-4ef9-a951-f607776400a2"}]: dispatch 2024-09-16T02:01:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:56 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f31571-16de-4ef9-a951-f607776400a2"}]: dispatch 2024-09-16T02:01:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:56 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3f31571-16de-4ef9-a951-f607776400a2"}]': finished 2024-09-16T02:01:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:56 smithi103 ceph-mon[30398]: osdmap e61: 11 total, 10 up, 11 in 2024-09-16T02:01:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:01:56.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:56 smithi103 ceph-mon[30398]: from='client.? 172.21.15.136:0/1038567236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:01:56.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:56 smithi136 ceph-mon[35323]: from='client.? 172.21.15.136:0/4114173798' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f31571-16de-4ef9-a951-f607776400a2"}]: dispatch 2024-09-16T02:01:56.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:56 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f31571-16de-4ef9-a951-f607776400a2"}]: dispatch 2024-09-16T02:01:56.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:56 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3f31571-16de-4ef9-a951-f607776400a2"}]': finished 2024-09-16T02:01:56.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:56 smithi136 ceph-mon[35323]: osdmap e61: 11 total, 10 up, 11 in 2024-09-16T02:01:56.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:01:56.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:56 smithi136 ceph-mon[35323]: from='client.? 172.21.15.136:0/1038567236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:01:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:56 smithi172 ceph-mon[35340]: from='client.? 172.21.15.136:0/4114173798' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f31571-16de-4ef9-a951-f607776400a2"}]: dispatch 2024-09-16T02:01:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:56 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f31571-16de-4ef9-a951-f607776400a2"}]: dispatch 2024-09-16T02:01:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:56 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3f31571-16de-4ef9-a951-f607776400a2"}]': finished 2024-09-16T02:01:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:56 smithi172 ceph-mon[35340]: osdmap e61: 11 total, 10 up, 11 in 2024-09-16T02:01:56.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:01:56.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:56 smithi172 ceph-mon[35340]: from='client.? 172.21.15.136:0/1038567236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:01:57.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:57 smithi103 ceph-mon[30398]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:01:57.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:57 smithi136 ceph-mon[35323]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:01:57.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:57 smithi172 ceph-mon[35340]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:01:59.777 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:01:59 smithi136 ceph-mon[35323]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:01:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:01:59 smithi172 ceph-mon[35340]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:00.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:01:59 smithi103 ceph-mon[30398]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:01.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:01 smithi136 ceph-mon[35323]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:01.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:01 smithi172 ceph-mon[35340]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:02.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:01 smithi103 ceph-mon[30398]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:03.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:03 smithi136 ceph-mon[35323]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:03.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:03 smithi172 ceph-mon[35340]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:04.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:03 smithi103 ceph-mon[30398]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:04.771 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:04 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T02:02:04.772 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:04 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:04.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:04 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T02:02:04.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:04 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:05.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:04 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T02:02:05.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:04 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:05.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:05 smithi136 ceph-mon[35323]: Deploying daemon osd.10 on smithi136 2024-09-16T02:02:05.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:05 smithi136 ceph-mon[35323]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:05.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:05 smithi172 ceph-mon[35340]: Deploying daemon osd.10 on smithi136 2024-09-16T02:02:05.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:05 smithi172 ceph-mon[35340]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:06.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:05 smithi103 ceph-mon[30398]: Deploying daemon osd.10 on smithi136 2024-09-16T02:02:06.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:05 smithi103 ceph-mon[30398]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:07.570 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:07 smithi136 ceph-mon[35323]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:07.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:07 smithi172 ceph-mon[35340]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:08.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:07 smithi103 ceph-mon[30398]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:09.677 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:09 smithi136 ceph-mon[35323]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:09.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:09 smithi172 ceph-mon[35340]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:10.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:09 smithi103 ceph-mon[30398]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:10.762 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:10 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:10.762 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:10 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:10.762 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:10 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:10.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:10 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:10.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:10 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:10.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:10 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:11.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:10 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:11.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:10 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:11.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:10 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:11.723 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:11 smithi136 ceph-mon[35323]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:11 smithi172 ceph-mon[35340]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:12.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:11 smithi103 ceph-mon[30398]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:12.844 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 10 on host 'smithi136' 2024-09-16T02:02:12.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.843+0000 7f64477fe640 1 -- 172.21.15.136:0/3192416179 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6458062630 con 0x7f64400a32c0 2024-09-16T02:02:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 -- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f64400a32c0 msgr2=0x7f64400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f64400a32c0 0x7f64400a5780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f64480058c0 tx=0x7f6448005850 comp rx=0 tx=0).stop 2024-09-16T02:02:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 -- 172.21.15.136:0/3192416179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6458100ad0 msgr2=0x7f645806aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6458100ad0 0x7f645806aad0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c970 tx=0x7f644c00ce30 comp rx=0 tx=0).stop 2024-09-16T02:02:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 -- 172.21.15.136:0/3192416179 shutdown_connections 2024-09-16T02:02:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f64400a32c0 0x7f64400a5780 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6458102690 0x7f645819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6458101cd0 0x7f645819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:12.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 --2- 172.21.15.136:0/3192416179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6458100ad0 0x7f645806aad0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:12.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 -- 172.21.15.136:0/3192416179 >> 172.21.15.136:0/3192416179 conn(0x7f64580fc280 msgr2=0x7f6458105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:12.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 -- 172.21.15.136:0/3192416179 shutdown_connections 2024-09-16T02:02:12.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T02:02:12.845+0000 7f645cf8c640 1 -- 172.21.15.136:0/3192416179 wait complete. 2024-09-16T02:02:13.451 DEBUG:teuthology.orchestra.run.smithi136:osd.10> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.10.service 2024-09-16T02:02:13.454 INFO:tasks.cephadm:Deploying osd.11 on smithi172 with /dev/vg_nvme/lv_1... 2024-09-16T02:02:13.454 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T02:02:13.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:13 smithi136 ceph-mon[35323]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:13.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:13.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:13 smithi172 ceph-mon[35340]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:13.982 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:02:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:13 smithi103 ceph-mon[30398]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:14.914 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:14 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:14.914 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:14 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:14.914 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:14 smithi136 ceph-mon[35323]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:15.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:14 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:15.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:14 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:15.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:14 smithi103 ceph-mon[30398]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:15.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:14 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:15.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:14 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:15.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:14 smithi172 ceph-mon[35340]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:16.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:15 smithi172 ceph-mon[35340]: from='osd.10 [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T02:02:16.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:15 smithi172 ceph-mon[35340]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T02:02:16.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:15 smithi103 ceph-mon[30398]: from='osd.10 [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T02:02:16.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:15 smithi103 ceph-mon[30398]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T02:02:16.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:15 smithi136 ceph-mon[35323]: from='osd.10 [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T02:02:16.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:15 smithi136 ceph-mon[35323]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T02:02:16.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:02:15 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:02:15.844+0000 7f6a560e2640 -1 osd.10 0 log_to_monitors true 2024-09-16T02:02:16.680 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-16T02:02:16.708 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_1 2024-09-16T02:02:16.969 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:02:17.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T02:02:17.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='osd.10 [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: osdmap e62: 11 total, 10 up, 11 in 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi136 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi136 to 4272M 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: osdmap e63: 11 total, 10 up, 11 in 2024-09-16T02:02:17.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:17.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='osd.10 [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: osdmap e62: 11 total, 10 up, 11 in 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi136 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi136 to 4272M 2024-09-16T02:02:17.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:17.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:17.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:17.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:17.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:17.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T02:02:17.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: osdmap e63: 11 total, 10 up, 11 in 2024-09-16T02:02:17.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:17.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:02:16.909+0000 7f6a51842640 -1 osd.10 0 waiting for initial osdmap 2024-09-16T02:02:17.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:02:16.921+0000 7f6a4de7c640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T02:02:17.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T02:02:17.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='osd.10 [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:02:17.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: osdmap e62: 11 total, 10 up, 11 in 2024-09-16T02:02:17.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:17.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi136 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi136 to 4272M 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.275 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:17.275 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:17.275 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:17.275 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:17.275 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-16T02:02:17.275 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: osdmap e63: 11 total, 10 up, 11 in 2024-09-16T02:02:17.275 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:17.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.403+0000 7fb967673640 1 -- 172.21.15.172:0/854206508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb960101490 msgr2=0x7fb960101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:17.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.403+0000 7fb967673640 1 --2- 172.21.15.172:0/854206508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb960101490 0x7fb960101910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb95000aa10 tx=0x7fb95002b720 comp rx=0 tx=0).stop 2024-09-16T02:02:17.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.403+0000 7fb967673640 1 -- 172.21.15.172:0/854206508 shutdown_connections 2024-09-16T02:02:17.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.403+0000 7fb967673640 1 --2- 172.21.15.172:0/854206508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb960101e50 0x7fb9601086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:17.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.403+0000 7fb967673640 1 --2- 172.21.15.172:0/854206508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb960101490 0x7fb960101910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:17.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.403+0000 7fb967673640 1 --2- 172.21.15.172:0/854206508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb960100290 0x7fb960100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:17.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.403+0000 7fb967673640 1 -- 172.21.15.172:0/854206508 >> 172.21.15.172:0/854206508 conn(0x7fb9600fba40 msgr2=0x7fb9600fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:17.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.404+0000 7fb967673640 1 -- 172.21.15.172:0/854206508 shutdown_connections 2024-09-16T02:02:17.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.404+0000 7fb967673640 1 -- 172.21.15.172:0/854206508 wait complete. 2024-09-16T02:02:17.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.404+0000 7fb967673640 1 Processor -- start 2024-09-16T02:02:17.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.404+0000 7fb967673640 1 -- start start 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.404+0000 7fb967673640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb960100290 0x7fb960197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.404+0000 7fb967673640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb960101490 0x7fb960198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.404+0000 7fb967673640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb960101e50 0x7fb96019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.404+0000 7fb967673640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb96010b6b0 con 0x7fb960101e50 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.404+0000 7fb967673640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb96010b530 con 0x7fb960101490 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.404+0000 7fb967673640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb96010b830 con 0x7fb960100290 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb96540f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb960100290 0x7fb960197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb96540f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb960100290 0x7fb960197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59926/0 (socket says 172.21.15.172:59926) 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb96540f640 1 -- 172.21.15.172:0/2121530676 learned_addr learned my addr 172.21.15.172:0/2121530676 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb964c0e640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb960101490 0x7fb960198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb965c10640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb960101e50 0x7fb96019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:17.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb96540f640 1 -- 172.21.15.172:0/2121530676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb960101490 msgr2=0x7fb960198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb96540f640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb960101490 0x7fb960198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb96540f640 1 -- 172.21.15.172:0/2121530676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb960101e50 msgr2=0x7fb96019f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb96540f640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb960101e50 0x7fb96019f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb96540f640 1 -- 172.21.15.172:0/2121530676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96019fb30 con 0x7fb960100290 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb96540f640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb960100290 0x7fb960197f00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00ccb0 tx=0x7fb95c00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb9567fc640 1 -- 172.21.15.172:0/2121530676 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c003b70 con 0x7fb960100290 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb967673640 1 -- 172.21.15.172:0/2121530676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96019fe20 con 0x7fb960100290 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb9567fc640 1 -- 172.21.15.172:0/2121530676 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb95c00ae50 con 0x7fb960100290 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb9567fc640 1 -- 172.21.15.172:0/2121530676 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c00f810 con 0x7fb960100290 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.405+0000 7fb967673640 1 -- 172.21.15.172:0/2121530676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9601a0270 con 0x7fb960100290 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.406+0000 7fb967673640 1 -- 172.21.15.172:0/2121530676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb960107660 con 0x7fb960100290 2024-09-16T02:02:17.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.408+0000 7fb9567fc640 1 -- 172.21.15.172:0/2121530676 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb95c00f9b0 con 0x7fb960100290 2024-09-16T02:02:17.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.408+0000 7fb9567fc640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb94c0a3270 0x7fb94c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:17.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.408+0000 7fb9567fc640 1 -- 172.21.15.172:0/2121530676 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6414+0+0 (secure 0 0 0) 0x7fb95c0cf5e0 con 0x7fb960100290 2024-09-16T02:02:17.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.409+0000 7fb964c0e640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb94c0a3270 0x7fb94c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:17.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.409+0000 7fb964c0e640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb94c0a3270 0x7fb94c0a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb950000c00 tx=0x7fb9500306c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:17.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.412+0000 7fb9567fc640 1 -- 172.21.15.172:0/2121530676 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb95c014070 con 0x7fb960100290 2024-09-16T02:02:17.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:17.587+0000 7fb967673640 1 -- 172.21.15.172:0/2121530676 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb960062630 con 0x7fb94c0a3270 2024-09-16T02:02:18.206 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:18.207 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:17 smithi172 ceph-mon[35340]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:18.207 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:02:18.207 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:02:18.207 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:18.207 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:17 smithi172 ceph-mon[35340]: osd.10 [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] boot 2024-09-16T02:02:18.207 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:17 smithi172 ceph-mon[35340]: osdmap e64: 11 total, 11 up, 11 in 2024-09-16T02:02:18.207 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:17 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:18.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:18.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:17 smithi103 ceph-mon[30398]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:18.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:02:18.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:02:18.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:18.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:17 smithi103 ceph-mon[30398]: osd.10 [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] boot 2024-09-16T02:02:18.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:17 smithi103 ceph-mon[30398]: osdmap e64: 11 total, 11 up, 11 in 2024-09-16T02:02:18.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:17 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:18.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:18.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:17 smithi136 ceph-mon[35323]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:18.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T02:02:18.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T02:02:18.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:18.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:17 smithi136 ceph-mon[35323]: osd.10 [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] boot 2024-09-16T02:02:18.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:17 smithi136 ceph-mon[35323]: osdmap e64: 11 total, 11 up, 11 in 2024-09-16T02:02:18.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:17 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T02:02:19.676 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:19 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T02:02:19.676 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:19 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T02:02:19.676 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:19 smithi172 ceph-mon[35340]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:19.677 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:19 smithi172 ceph-mon[35340]: osdmap e65: 11 total, 11 up, 11 in 2024-09-16T02:02:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:19 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T02:02:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:19 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T02:02:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:19 smithi136 ceph-mon[35323]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:19 smithi136 ceph-mon[35323]: osdmap e65: 11 total, 11 up, 11 in 2024-09-16T02:02:20.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:19 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T02:02:20.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:19 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T02:02:20.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:19 smithi103 ceph-mon[30398]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:20.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:19 smithi103 ceph-mon[30398]: osdmap e65: 11 total, 11 up, 11 in 2024-09-16T02:02:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:20 smithi136 ceph-mon[35323]: osdmap e66: 11 total, 11 up, 11 in 2024-09-16T02:02:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:20 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/3743297181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84a838c3-7bec-402f-a65a-78030b0380d7"}]: dispatch 2024-09-16T02:02:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:20 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84a838c3-7bec-402f-a65a-78030b0380d7"}]: dispatch 2024-09-16T02:02:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:20 smithi136 ceph-mon[35323]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84a838c3-7bec-402f-a65a-78030b0380d7"}]': finished 2024-09-16T02:02:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:20 smithi136 ceph-mon[35323]: osdmap e67: 12 total, 11 up, 12 in 2024-09-16T02:02:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:20 smithi172 ceph-mon[35340]: osdmap e66: 11 total, 11 up, 11 in 2024-09-16T02:02:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:20 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/3743297181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84a838c3-7bec-402f-a65a-78030b0380d7"}]: dispatch 2024-09-16T02:02:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:20 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84a838c3-7bec-402f-a65a-78030b0380d7"}]: dispatch 2024-09-16T02:02:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:20 smithi172 ceph-mon[35340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84a838c3-7bec-402f-a65a-78030b0380d7"}]': finished 2024-09-16T02:02:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:20 smithi172 ceph-mon[35340]: osdmap e67: 12 total, 11 up, 12 in 2024-09-16T02:02:20.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:21.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:20 smithi103 ceph-mon[30398]: osdmap e66: 11 total, 11 up, 11 in 2024-09-16T02:02:21.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:20 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/3743297181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84a838c3-7bec-402f-a65a-78030b0380d7"}]: dispatch 2024-09-16T02:02:21.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:20 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84a838c3-7bec-402f-a65a-78030b0380d7"}]: dispatch 2024-09-16T02:02:21.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:20 smithi103 ceph-mon[30398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84a838c3-7bec-402f-a65a-78030b0380d7"}]': finished 2024-09-16T02:02:21.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:20 smithi103 ceph-mon[30398]: osdmap e67: 12 total, 11 up, 12 in 2024-09-16T02:02:21.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:21 smithi136 ceph-mon[35323]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:21 smithi136 ceph-mon[35323]: from='client.? 172.21.15.172:0/3174370886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:02:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:21 smithi136 ceph-mon[35323]: osdmap e68: 12 total, 11 up, 12 in 2024-09-16T02:02:21.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:21 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:21.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:21 smithi172 ceph-mon[35340]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:21.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:21 smithi172 ceph-mon[35340]: from='client.? 172.21.15.172:0/3174370886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:02:21.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:21 smithi172 ceph-mon[35340]: osdmap e68: 12 total, 11 up, 12 in 2024-09-16T02:02:21.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:21 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:21 smithi103 ceph-mon[30398]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:21 smithi103 ceph-mon[30398]: from='client.? 172.21.15.172:0/3174370886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T02:02:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:21 smithi103 ceph-mon[30398]: osdmap e68: 12 total, 11 up, 12 in 2024-09-16T02:02:22.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:21 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:23.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:23 smithi136 ceph-mon[35323]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T02:02:23.794 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:23 smithi172 ceph-mon[35340]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T02:02:24.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:23 smithi103 ceph-mon[30398]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T02:02:25.734 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:25 smithi172 ceph-mon[35340]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T02:02:26.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:25 smithi103 ceph-mon[30398]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T02:02:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:25 smithi136 ceph-mon[35323]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T02:02:27.672 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:27 smithi172 ceph-mon[35340]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T02:02:28.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:27 smithi103 ceph-mon[30398]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T02:02:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:27 smithi136 ceph-mon[35323]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T02:02:29.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:29 smithi172 ceph-mon[35340]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T02:02:29.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T02:02:29.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:30.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:29 smithi103 ceph-mon[30398]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T02:02:30.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T02:02:30.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:29 smithi136 ceph-mon[35323]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T02:02:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T02:02:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:30.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:30 smithi172 ceph-mon[35340]: Deploying daemon osd.11 on smithi172 2024-09-16T02:02:31.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:30 smithi103 ceph-mon[30398]: Deploying daemon osd.11 on smithi172 2024-09-16T02:02:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:30 smithi136 ceph-mon[35323]: Deploying daemon osd.11 on smithi172 2024-09-16T02:02:31.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:31 smithi172 ceph-mon[35340]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T02:02:32.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:31 smithi103 ceph-mon[30398]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T02:02:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:31 smithi136 ceph-mon[35323]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T02:02:33.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:33 smithi172 ceph-mon[35340]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:34.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:33 smithi103 ceph-mon[30398]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:33 smithi136 ceph-mon[35323]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:34.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:34 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:34.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:34 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:34.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:34 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:35.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:34 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:35.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:34 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:35.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:34 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:34 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:34 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:34 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:35.872 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:35 smithi172 ceph-mon[35340]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:36.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:35 smithi103 ceph-mon[30398]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:36.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:35 smithi136 ceph-mon[35323]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:37.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:37 smithi172 ceph-mon[35340]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:37.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:37 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:37.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:37 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:37.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:37 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:37.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:37 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:37.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:37 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 11 on host 'smithi172' 2024-09-16T02:02:37.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.884+0000 7fb9567fc640 1 -- 172.21.15.172:0/2121530676 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb960062630 con 0x7fb94c0a3270 2024-09-16T02:02:37.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 -- 172.21.15.172:0/2121530676 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb94c0a3270 msgr2=0x7fb94c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:37.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb94c0a3270 0x7fb94c0a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb950000c00 tx=0x7fb9500306c0 comp rx=0 tx=0).stop 2024-09-16T02:02:37.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 -- 172.21.15.172:0/2121530676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb960100290 msgr2=0x7fb960197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:37.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb960100290 0x7fb960197f00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00ccb0 tx=0x7fb95c00a620 comp rx=0 tx=0).stop 2024-09-16T02:02:37.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 -- 172.21.15.172:0/2121530676 shutdown_connections 2024-09-16T02:02:37.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb94c0a3270 0x7fb94c0a5730 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:37.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb960101e50 0x7fb96019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:37.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb960101490 0x7fb960198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:37.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 --2- 172.21.15.172:0/2121530676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb960100290 0x7fb960197f00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:37.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 -- 172.21.15.172:0/2121530676 >> 172.21.15.172:0/2121530676 conn(0x7fb9600fba40 msgr2=0x7fb960104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:37.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 -- 172.21.15.172:0/2121530676 shutdown_connections 2024-09-16T02:02:37.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:37.887+0000 7fb967673640 1 -- 172.21.15.172:0/2121530676 wait complete. 2024-09-16T02:02:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:37 smithi103 ceph-mon[30398]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:37 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:37 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:37 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:37 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:37 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:37 smithi136 ceph-mon[35323]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:37 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:37 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:37 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:37 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:37 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:38.458 DEBUG:teuthology.orchestra.run.smithi172:osd.11> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.11.service 2024-09-16T02:02:38.461 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-16T02:02:38.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T02:02:38.777 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:38 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:38.777 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:38 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:38.777 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:38 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:38.838 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:38.977 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:38 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:38.977 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:38 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:38.977 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:38 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:38 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:38 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:38 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.345+0000 7f4a8aa00640 1 -- 172.21.15.103:0/3844829618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a84101c00 msgr2=0x7f4a84102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.345+0000 7f4a8aa00640 1 --2- 172.21.15.103:0/3844829618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a84101c00 0x7f4a84102000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009900 tx=0x7f4a7402a410 comp rx=0 tx=0).stop 2024-09-16T02:02:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.346+0000 7f4a8aa00640 1 -- 172.21.15.103:0/3844829618 shutdown_connections 2024-09-16T02:02:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.346+0000 7f4a8aa00640 1 --2- 172.21.15.103:0/3844829618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a841037c0 0x7f4a8410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.346+0000 7f4a8aa00640 1 --2- 172.21.15.103:0/3844829618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a84102e00 0x7f4a84103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.346+0000 7f4a8aa00640 1 --2- 172.21.15.103:0/3844829618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a84101c00 0x7f4a84102000 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.346+0000 7f4a8aa00640 1 -- 172.21.15.103:0/3844829618 >> 172.21.15.103:0/3844829618 conn(0x7f4a840fd390 msgr2=0x7f4a840ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.346+0000 7f4a8aa00640 1 -- 172.21.15.103:0/3844829618 shutdown_connections 2024-09-16T02:02:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.346+0000 7f4a8aa00640 1 -- 172.21.15.103:0/3844829618 wait complete. 2024-09-16T02:02:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.346+0000 7f4a8aa00640 1 Processor -- start 2024-09-16T02:02:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a8aa00640 1 -- start start 2024-09-16T02:02:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a8aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a84101c00 0x7f4a8419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a8aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a84102e00 0x7f4a8419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a8aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a841037c0 0x7f4a841a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a8aa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8410d9b0 con 0x7f4a84102e00 2024-09-16T02:02:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a8aa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8410d830 con 0x7f4a84101c00 2024-09-16T02:02:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a8aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8410db30 con 0x7f4a841037c0 2024-09-16T02:02:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a83400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a84102e00 0x7f4a8419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a83400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a84102e00 0x7f4a8419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39790/0 (socket says 172.21.15.103:39790) 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a83400640 1 -- 172.21.15.103:0/2682322939 learned_addr learned my addr 172.21.15.103:0/2682322939 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a83e00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a84101c00 0x7f4a8419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.347+0000 7f4a88a00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a841037c0 0x7f4a841a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.348+0000 7f4a83400640 1 -- 172.21.15.103:0/2682322939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a841037c0 msgr2=0x7f4a841a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.348+0000 7f4a83400640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a841037c0 0x7f4a841a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.348+0000 7f4a83400640 1 -- 172.21.15.103:0/2682322939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a84101c00 msgr2=0x7f4a8419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.348+0000 7f4a83400640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a84101c00 0x7f4a8419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.348+0000 7f4a83400640 1 -- 172.21.15.103:0/2682322939 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a841a1ed0 con 0x7f4a84102e00 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.348+0000 7f4a83400640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a84102e00 0x7f4a8419a7e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000b4a0 tx=0x7f4a7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.348+0000 7f4a80c00640 1 -- 172.21.15.103:0/2682322939 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70014070 con 0x7f4a84102e00 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.348+0000 7f4a80c00640 1 -- 172.21.15.103:0/2682322939 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a70010af0 con 0x7f4a84102e00 2024-09-16T02:02:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.348+0000 7f4a8aa00640 1 -- 172.21.15.103:0/2682322939 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a841a21c0 con 0x7f4a84102e00 2024-09-16T02:02:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.348+0000 7f4a80c00640 1 -- 172.21.15.103:0/2682322939 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70017d10 con 0x7f4a84102e00 2024-09-16T02:02:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.349+0000 7f4a8aa00640 1 -- 172.21.15.103:0/2682322939 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a841a2780 con 0x7f4a84102e00 2024-09-16T02:02:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.350+0000 7f4a8aa00640 1 -- 172.21.15.103:0/2682322939 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a841a2a40 con 0x7f4a84102e00 2024-09-16T02:02:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.352+0000 7f4a80c00640 1 -- 172.21.15.103:0/2682322939 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4a70017070 con 0x7f4a84102e00 2024-09-16T02:02:39.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.355+0000 7f4a80c00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4a680a3270 0x7f4a680a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:39.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.355+0000 7f4a80c00640 1 -- 172.21.15.103:0/2682322939 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6809+0+0 (secure 0 0 0) 0x7f4a700cea40 con 0x7f4a84102e00 2024-09-16T02:02:39.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.355+0000 7f4a83e00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4a680a3270 0x7f4a680a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:39.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.356+0000 7f4a83e00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4a680a3270 0x7f4a680a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4a7402aa00 tx=0x7f4a740315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:39.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.357+0000 7f4a80c00640 1 -- 172.21.15.103:0/2682322939 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a7009b650 con 0x7f4a84102e00 2024-09-16T02:02:39.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.532+0000 7f4a8aa00640 1 -- 172.21.15.103:0/2682322939 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4a841a2c10 con 0x7f4a84102e00 2024-09-16T02:02:39.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.533+0000 7f4a80c00640 1 -- 172.21.15.103:0/2682322939 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f4a7009f950 con 0x7f4a84102e00 2024-09-16T02:02:39.535 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:02:39.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 -- 172.21.15.103:0/2682322939 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4a680a3270 msgr2=0x7f4a680a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:39.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4a680a3270 0x7f4a680a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4a7402aa00 tx=0x7f4a740315b0 comp rx=0 tx=0).stop 2024-09-16T02:02:39.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 -- 172.21.15.103:0/2682322939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a84102e00 msgr2=0x7f4a8419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:39.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a84102e00 0x7f4a8419a7e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000b4a0 tx=0x7f4a7000b960 comp rx=0 tx=0).stop 2024-09-16T02:02:39.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 -- 172.21.15.103:0/2682322939 shutdown_connections 2024-09-16T02:02:39.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4a680a3270 0x7f4a680a5730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:39.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a841037c0 0x7f4a841a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:39.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a84102e00 0x7f4a8419a7e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:39.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 --2- 172.21.15.103:0/2682322939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a84101c00 0x7f4a8419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:39.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 -- 172.21.15.103:0/2682322939 >> 172.21.15.103:0/2682322939 conn(0x7f4a840fd390 msgr2=0x7f4a84108610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:39.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 -- 172.21.15.103:0/2682322939 shutdown_connections 2024-09-16T02:02:39.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:39.535+0000 7f4a8aa00640 1 -- 172.21.15.103:0/2682322939 wait complete. 2024-09-16T02:02:39.845 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:39 smithi172 ceph-mon[35340]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:39.845 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:39 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2682322939' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T02:02:39.924 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:39 smithi136 ceph-mon[35323]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:39.925 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:39 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2682322939' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T02:02:39.937 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:39 smithi103 ceph-mon[30398]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:39.937 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:39 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2682322939' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T02:02:40.319 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726452137,"num_in_osds":12,"osd_in_since":1726452139,"num_remapped_pgs":0} 2024-09-16T02:02:40.984 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:02:40 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:02:40.809+0000 7fe94a0d2640 -1 osd.11 0 log_to_monitors true 2024-09-16T02:02:41.320 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T02:02:41.678 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:41.678 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='osd.11 [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T02:02:41.678 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T02:02:41.678 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:41.678 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:41.678 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:41.678 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:41.679 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:41.679 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:41.679 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:41.679 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:41.679 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:41.679 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:41 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:41.721 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:42.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:42.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='osd.11 [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T02:02:42.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T02:02:42.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:42.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:42.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:42.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:42.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:42.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:42.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:42.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:42.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:42.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:41 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:42.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:42.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='osd.11 [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T02:02:42.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T02:02:42.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:42.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:42.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:42.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:42.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:42.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T02:02:42.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:42.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:42.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:42.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:41 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.204+0000 7f4b8d000640 1 -- 172.21.15.103:0/1030627437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b88074bf0 msgr2=0x7f4b88075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.204+0000 7f4b8d000640 1 --2- 172.21.15.103:0/1030627437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b88074bf0 0x7f4b88075050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009900 tx=0x7f4b7402a4a0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.204+0000 7f4b8d000640 1 -- 172.21.15.103:0/1030627437 shutdown_connections 2024-09-16T02:02:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.204+0000 7f4b8d000640 1 --2- 172.21.15.103:0/1030627437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b881098e0 0x7f4b8810bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.204+0000 7f4b8d000640 1 --2- 172.21.15.103:0/1030627437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b88074bf0 0x7f4b88075050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.204+0000 7f4b8d000640 1 --2- 172.21.15.103:0/1030627437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b880761c0 0x7f4b88074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.204+0000 7f4b8d000640 1 -- 172.21.15.103:0/1030627437 >> 172.21.15.103:0/1030627437 conn(0x7f4b880fd1f0 msgr2=0x7f4b880ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.204+0000 7f4b8d000640 1 -- 172.21.15.103:0/1030627437 shutdown_connections 2024-09-16T02:02:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.204+0000 7f4b8d000640 1 -- 172.21.15.103:0/1030627437 wait complete. 2024-09-16T02:02:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.205+0000 7f4b8d000640 1 Processor -- start 2024-09-16T02:02:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.205+0000 7f4b8d000640 1 -- start start 2024-09-16T02:02:42.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.206+0000 7f4b8d000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b88074bf0 0x7f4b88199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.206+0000 7f4b8d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b880761c0 0x7f4b8819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.206+0000 7f4b8d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b881098e0 0x7f4b881a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.206+0000 7f4b8d000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810f990 con 0x7f4b880761c0 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.206+0000 7f4b8d000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810f810 con 0x7f4b881098e0 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.206+0000 7f4b8d000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810fb10 con 0x7f4b88074bf0 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.206+0000 7f4b86a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b880761c0 0x7f4b8819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.206+0000 7f4b86a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b880761c0 0x7f4b8819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39800/0 (socket says 172.21.15.103:39800) 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.206+0000 7f4b86a00640 1 -- 172.21.15.103:0/1663348756 learned_addr learned my addr 172.21.15.103:0/1663348756 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.207+0000 7f4b86a00640 1 -- 172.21.15.103:0/1663348756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b88074bf0 msgr2=0x7f4b88199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.207+0000 7f4b86a00640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b88074bf0 0x7f4b88199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.207+0000 7f4b86a00640 1 -- 172.21.15.103:0/1663348756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b881098e0 msgr2=0x7f4b881a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.207+0000 7f4b86a00640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b881098e0 0x7f4b881a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.207+0000 7f4b86a00640 1 -- 172.21.15.103:0/1663348756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b881a1c20 con 0x7f4b880761c0 2024-09-16T02:02:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.207+0000 7f4b86a00640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b880761c0 0x7f4b8819a530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b74031440 tx=0x7f4b74032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:42.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.207+0000 7f4b7fe00640 1 -- 172.21.15.103:0/1663348756 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7403f030 con 0x7f4b880761c0 2024-09-16T02:02:42.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.207+0000 7f4b7fe00640 1 -- 172.21.15.103:0/1663348756 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b74003ce0 con 0x7f4b880761c0 2024-09-16T02:02:42.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.207+0000 7f4b8d000640 1 -- 172.21.15.103:0/1663348756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b881a1eb0 con 0x7f4b880761c0 2024-09-16T02:02:42.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.208+0000 7f4b7fe00640 1 -- 172.21.15.103:0/1663348756 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74035070 con 0x7f4b880761c0 2024-09-16T02:02:42.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.208+0000 7f4b8d000640 1 -- 172.21.15.103:0/1663348756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a23e0 con 0x7f4b880761c0 2024-09-16T02:02:42.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.209+0000 7f4b8d000640 1 -- 172.21.15.103:0/1663348756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b881086a0 con 0x7f4b880761c0 2024-09-16T02:02:42.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.211+0000 7f4b7fe00640 1 -- 172.21.15.103:0/1663348756 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4b74042030 con 0x7f4b880761c0 2024-09-16T02:02:42.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.212+0000 7f4b7fe00640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4b6c0a32f0 0x7f4b6c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:42.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.212+0000 7f4b7fe00640 1 -- 172.21.15.103:0/1663348756 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6831+0+0 (secure 0 0 0) 0x7f4b740ef980 con 0x7f4b880761c0 2024-09-16T02:02:42.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.212+0000 7f4b87400640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4b6c0a32f0 0x7f4b6c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:42.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.213+0000 7f4b87400640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4b6c0a32f0 0x7f4b6c0a57b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b780096f0 tx=0x7f4b78006c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.216+0000 7f4b7fe00640 1 -- 172.21.15.103:0/1663348756 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b740bcbd0 con 0x7f4b880761c0 2024-09-16T02:02:42.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.417+0000 7f4b8d000640 1 -- 172.21.15.103:0/1663348756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4b88062ef0 con 0x7f4b880761c0 2024-09-16T02:02:42.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.418+0000 7f4b7fe00640 1 -- 172.21.15.103:0/1663348756 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7f4b740bc4d0 con 0x7f4b880761c0 2024-09-16T02:02:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:02:42.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 -- 172.21.15.103:0/1663348756 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4b6c0a32f0 msgr2=0x7f4b6c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4b6c0a32f0 0x7f4b6c0a57b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b780096f0 tx=0x7f4b78006c40 comp rx=0 tx=0).stop 2024-09-16T02:02:42.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 -- 172.21.15.103:0/1663348756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b880761c0 msgr2=0x7f4b8819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:42.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b880761c0 0x7f4b8819a530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b74031440 tx=0x7f4b74032ce0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 -- 172.21.15.103:0/1663348756 shutdown_connections 2024-09-16T02:02:42.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4b6c0a32f0 0x7f4b6c0a57b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b881098e0 0x7f4b881a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b880761c0 0x7f4b8819a530 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 --2- 172.21.15.103:0/1663348756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b88074bf0 0x7f4b88199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:42.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 -- 172.21.15.103:0/1663348756 >> 172.21.15.103:0/1663348756 conn(0x7f4b880fd1f0 msgr2=0x7f4b88105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:42.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 -- 172.21.15.103:0/1663348756 shutdown_connections 2024-09-16T02:02:42.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:42.420+0000 7f4b8d000640 1 -- 172.21.15.103:0/1663348756 wait complete. 2024-09-16T02:02:42.727 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:42 smithi103 ceph-mon[30398]: Detected new or changed devices on smithi172 2024-09-16T02:02:42.727 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:42 smithi103 ceph-mon[30398]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-16T02:02:42.728 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:42 smithi103 ceph-mon[30398]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T02:02:42.728 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:42 smithi103 ceph-mon[30398]: osdmap e69: 12 total, 11 up, 12 in 2024-09-16T02:02:42.728 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:42 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:42.728 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:42 smithi103 ceph-mon[30398]: from='osd.11 [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:02:42.728 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:42 smithi103 ceph-mon[30398]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:02:42.728 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:42 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1663348756' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T02:02:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:42 smithi136 ceph-mon[35323]: Detected new or changed devices on smithi172 2024-09-16T02:02:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:42 smithi136 ceph-mon[35323]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-16T02:02:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:42 smithi136 ceph-mon[35323]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T02:02:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:42 smithi136 ceph-mon[35323]: osdmap e69: 12 total, 11 up, 12 in 2024-09-16T02:02:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:42 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:42 smithi136 ceph-mon[35323]: from='osd.11 [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:02:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:42 smithi136 ceph-mon[35323]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:02:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:42 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1663348756' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T02:02:43.045 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:02:42 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:02:42.572+0000 7fe945832640 -1 osd.11 0 waiting for initial osdmap 2024-09-16T02:02:43.045 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:02:42 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:02:42.587+0000 7fe941e6c640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T02:02:43.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:42 smithi172 ceph-mon[35340]: Detected new or changed devices on smithi172 2024-09-16T02:02:43.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:42 smithi172 ceph-mon[35340]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-16T02:02:43.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:42 smithi172 ceph-mon[35340]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T02:02:43.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:42 smithi172 ceph-mon[35340]: osdmap e69: 12 total, 11 up, 12 in 2024-09-16T02:02:43.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:42 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:43.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:42 smithi172 ceph-mon[35340]: from='osd.11 [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:02:43.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:42 smithi172 ceph-mon[35340]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-16T02:02:43.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:42 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1663348756' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T02:02:43.201 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":69,"num_osds":12,"num_up_osds":11,"osd_up_since":1726452137,"num_in_osds":12,"osd_in_since":1726452139,"num_remapped_pgs":0} 2024-09-16T02:02:44.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:43 smithi103 ceph-mon[30398]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:44.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:43 smithi103 ceph-mon[30398]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T02:02:44.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:43 smithi103 ceph-mon[30398]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T02:02:44.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:43 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:44.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:43 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:44.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:43 smithi136 ceph-mon[35323]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:44.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:43 smithi136 ceph-mon[35323]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T02:02:44.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:43 smithi136 ceph-mon[35323]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T02:02:44.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:43 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:44.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:43 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:44.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:43 smithi172 ceph-mon[35340]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:44.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:43 smithi172 ceph-mon[35340]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-16T02:02:44.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:43 smithi172 ceph-mon[35340]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T02:02:44.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:43 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:44.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:43 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:44.202 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T02:02:44.582 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:44.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:44 smithi103 ceph-mon[30398]: purged_snaps scrub starts 2024-09-16T02:02:44.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:44 smithi103 ceph-mon[30398]: purged_snaps scrub ok 2024-09-16T02:02:44.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:44 smithi103 ceph-mon[30398]: osd.11 [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] boot 2024-09-16T02:02:44.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:44 smithi103 ceph-mon[30398]: osdmap e71: 12 total, 12 up, 12 in 2024-09-16T02:02:44.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:44 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:45.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:44 smithi136 ceph-mon[35323]: purged_snaps scrub starts 2024-09-16T02:02:45.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:44 smithi136 ceph-mon[35323]: purged_snaps scrub ok 2024-09-16T02:02:45.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:44 smithi136 ceph-mon[35323]: osd.11 [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] boot 2024-09-16T02:02:45.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:44 smithi136 ceph-mon[35323]: osdmap e71: 12 total, 12 up, 12 in 2024-09-16T02:02:45.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:44 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:45.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:44 smithi172 ceph-mon[35340]: purged_snaps scrub starts 2024-09-16T02:02:45.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:44 smithi172 ceph-mon[35340]: purged_snaps scrub ok 2024-09-16T02:02:45.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:44 smithi172 ceph-mon[35340]: osd.11 [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] boot 2024-09-16T02:02:45.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:44 smithi172 ceph-mon[35340]: osdmap e71: 12 total, 12 up, 12 in 2024-09-16T02:02:45.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:44 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T02:02:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.117+0000 7f5af8a00640 1 -- 172.21.15.103:0/3960665707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af4104e50 msgr2=0x7f5af41052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.117+0000 7f5af8a00640 1 --2- 172.21.15.103:0/3960665707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af4104e50 0x7f5af41052d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009900 tx=0x7f5ae002a410 comp rx=0 tx=0).stop 2024-09-16T02:02:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.117+0000 7f5af8a00640 1 -- 172.21.15.103:0/3960665707 shutdown_connections 2024-09-16T02:02:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.117+0000 7f5af8a00640 1 --2- 172.21.15.103:0/3960665707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af4105810 0x7f5af410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.117+0000 7f5af8a00640 1 --2- 172.21.15.103:0/3960665707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af4104e50 0x7f5af41052d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.117+0000 7f5af8a00640 1 --2- 172.21.15.103:0/3960665707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af4103c50 0x7f5af4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.117+0000 7f5af8a00640 1 -- 172.21.15.103:0/3960665707 >> 172.21.15.103:0/3960665707 conn(0x7f5af40ff3e0 msgr2=0x7f5af4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.117+0000 7f5af8a00640 1 -- 172.21.15.103:0/3960665707 shutdown_connections 2024-09-16T02:02:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.117+0000 7f5af8a00640 1 -- 172.21.15.103:0/3960665707 wait complete. 2024-09-16T02:02:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.118+0000 7f5af8a00640 1 Processor -- start 2024-09-16T02:02:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.118+0000 7f5af8a00640 1 -- start start 2024-09-16T02:02:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.118+0000 7f5af8a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af4103c50 0x7f5af4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.118+0000 7f5af8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af4104e50 0x7f5af419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.118+0000 7f5af8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af4105810 0x7f5af41a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.118+0000 7f5af8a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5af410f920 con 0x7f5af4103c50 2024-09-16T02:02:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.118+0000 7f5af8a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5af410f7a0 con 0x7f5af4104e50 2024-09-16T02:02:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.118+0000 7f5af8a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5af410faa0 con 0x7f5af4105810 2024-09-16T02:02:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af4104e50 0x7f5af419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af4103c50 0x7f5af4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af4104e50 0x7f5af419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:51956/0 (socket says 172.21.15.103:51956) 2024-09-16T02:02:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af1e00640 1 -- 172.21.15.103:0/811186402 learned_addr learned my addr 172.21.15.103:0/811186402 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:02:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af3200640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af4105810 0x7f5af41a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af3200640 1 -- 172.21.15.103:0/811186402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af4104e50 msgr2=0x7f5af419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af3200640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af4104e50 0x7f5af419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af3200640 1 -- 172.21.15.103:0/811186402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af4103c50 msgr2=0x7f5af4199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af3200640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af4103c50 0x7f5af4199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af3200640 1 -- 172.21.15.103:0/811186402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af41a1b50 con 0x7f5af4105810 2024-09-16T02:02:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.119+0000 7f5af3200640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af4105810 0x7f5af41a14e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00da10 tx=0x7f5aec00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:45.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.120+0000 7f5aeb400640 1 -- 172.21.15.103:0/811186402 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00bd70 con 0x7f5af4105810 2024-09-16T02:02:45.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.120+0000 7f5aeb400640 1 -- 172.21.15.103:0/811186402 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aec003e50 con 0x7f5af4105810 2024-09-16T02:02:45.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.120+0000 7f5aeb400640 1 -- 172.21.15.103:0/811186402 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0106a0 con 0x7f5af4105810 2024-09-16T02:02:45.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.120+0000 7f5af8a00640 1 -- 172.21.15.103:0/811186402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af41a1e40 con 0x7f5af4105810 2024-09-16T02:02:45.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.120+0000 7f5af8a00640 1 -- 172.21.15.103:0/811186402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af41a22a0 con 0x7f5af4105810 2024-09-16T02:02:45.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.121+0000 7f5af8a00640 1 -- 172.21.15.103:0/811186402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af410b000 con 0x7f5af4105810 2024-09-16T02:02:45.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.123+0000 7f5aeb400640 1 -- 172.21.15.103:0/811186402 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5aec004b40 con 0x7f5af4105810 2024-09-16T02:02:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.124+0000 7f5aeb400640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5ad80a3270 0x7f5ad80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.125+0000 7f5aeb400640 1 -- 172.21.15.103:0/811186402 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f5aec0cfb80 con 0x7f5af4105810 2024-09-16T02:02:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.125+0000 7f5af2800640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5ad80a3270 0x7f5ad80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.125+0000 7f5af2800640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5ad80a3270 0x7f5ad80a5730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4005fd0 tx=0x7f5ae400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.127+0000 7f5aeb400640 1 -- 172.21.15.103:0/811186402 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5aec09cca0 con 0x7f5af4105810 2024-09-16T02:02:45.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.303+0000 7f5af8a00640 1 -- 172.21.15.103:0/811186402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5af402da10 con 0x7f5af4105810 2024-09-16T02:02:45.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.304+0000 7f5aeb400640 1 -- 172.21.15.103:0/811186402 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f5aec09c5a0 con 0x7f5af4105810 2024-09-16T02:02:45.306 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:02:45.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 -- 172.21.15.103:0/811186402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5ad80a3270 msgr2=0x7f5ad80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5ad80a3270 0x7f5ad80a5730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4005fd0 tx=0x7f5ae400b390 comp rx=0 tx=0).stop 2024-09-16T02:02:45.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 -- 172.21.15.103:0/811186402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af4105810 msgr2=0x7f5af41a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:45.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af4105810 0x7f5af41a14e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00da10 tx=0x7f5aec00ded0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 -- 172.21.15.103:0/811186402 shutdown_connections 2024-09-16T02:02:45.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5ad80a3270 0x7f5ad80a5730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af4105810 0x7f5af41a14e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5af4104e50 0x7f5af419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 --2- 172.21.15.103:0/811186402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af4103c50 0x7f5af4199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:45.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 -- 172.21.15.103:0/811186402 >> 172.21.15.103:0/811186402 conn(0x7f5af40ff3e0 msgr2=0x7f5af41081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:45.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.306+0000 7f5af8a00640 1 -- 172.21.15.103:0/811186402 shutdown_connections 2024-09-16T02:02:45.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:45.307+0000 7f5af8a00640 1 -- 172.21.15.103:0/811186402 wait complete. 2024-09-16T02:02:45.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:45 smithi103 ceph-mon[30398]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:45.930 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:45 smithi103 ceph-mon[30398]: osdmap e72: 12 total, 12 up, 12 in 2024-09-16T02:02:45.930 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:45 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/811186402' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T02:02:45.966 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1726452163,"num_in_osds":12,"osd_in_since":1726452139,"num_remapped_pgs":0} 2024-09-16T02:02:45.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:02:46.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:45 smithi136 ceph-mon[35323]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:46.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:45 smithi136 ceph-mon[35323]: osdmap e72: 12 total, 12 up, 12 in 2024-09-16T02:02:46.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:45 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/811186402' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T02:02:46.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:45 smithi172 ceph-mon[35340]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-16T02:02:46.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:45 smithi172 ceph-mon[35340]: osdmap e72: 12 total, 12 up, 12 in 2024-09-16T02:02:46.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:45 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/811186402' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T02:02:46.382 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:46.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.958+0000 7fba04200640 1 -- 172.21.15.103:0/220784532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9fc112a30 msgr2=0x7fb9fc114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:46.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.958+0000 7fba04200640 1 --2- 172.21.15.103:0/220784532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9fc112a30 0x7fb9fc114e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009ca0 tx=0x7fb9f402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.958+0000 7fba04200640 1 -- 172.21.15.103:0/220784532 shutdown_connections 2024-09-16T02:02:46.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.958+0000 7fba04200640 1 --2- 172.21.15.103:0/220784532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9fc112a30 0x7fb9fc114e20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.958+0000 7fba04200640 1 --2- 172.21.15.103:0/220784532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9fc077040 0x7fb9fc1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.958+0000 7fba04200640 1 --2- 172.21.15.103:0/220784532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9fc076720 0x7fb9fc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.958+0000 7fba04200640 1 -- 172.21.15.103:0/220784532 >> 172.21.15.103:0/220784532 conn(0x7fb9fc0ffa90 msgr2=0x7fb9fc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:46.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.958+0000 7fba04200640 1 -- 172.21.15.103:0/220784532 shutdown_connections 2024-09-16T02:02:46.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.958+0000 7fba04200640 1 -- 172.21.15.103:0/220784532 wait complete. 2024-09-16T02:02:46.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.959+0000 7fba04200640 1 Processor -- start 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.959+0000 7fba04200640 1 -- start start 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba04200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9fc076720 0x7fb9fc1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba04200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9fc077040 0x7fb9fc1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba04200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9fc112a30 0x7fb9fc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba04200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc118580 con 0x7fb9fc112a30 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba04200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc118400 con 0x7fb9fc076720 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba04200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc118700 con 0x7fb9fc077040 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba02200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9fc112a30 0x7fb9fc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba02200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9fc112a30 0x7fb9fc19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39856/0 (socket says 172.21.15.103:39856) 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba02200640 1 -- 172.21.15.103:0/4088610642 learned_addr learned my addr 172.21.15.103:0/4088610642 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:02:46.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba00e00640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9fc077040 0x7fb9fc1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:46.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba01800640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9fc076720 0x7fb9fc1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:46.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba02200640 1 -- 172.21.15.103:0/4088610642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9fc077040 msgr2=0x7fb9fc1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:46.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba02200640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9fc077040 0x7fb9fc1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba02200640 1 -- 172.21.15.103:0/4088610642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9fc076720 msgr2=0x7fb9fc1a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:46.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.960+0000 7fba02200640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9fc076720 0x7fb9fc1a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:46.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.961+0000 7fba02200640 1 -- 172.21.15.103:0/4088610642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9fc19efe0 con 0x7fb9fc112a30 2024-09-16T02:02:46.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.961+0000 7fba02200640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9fc112a30 0x7fb9fc19e840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9f402ade0 tx=0x7fb9f40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:46.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.961+0000 7fb9fa000640 1 -- 172.21.15.103:0/4088610642 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4003b70 con 0x7fb9fc112a30 2024-09-16T02:02:46.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.961+0000 7fb9fa000640 1 -- 172.21.15.103:0/4088610642 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f4003d10 con 0x7fb9fc112a30 2024-09-16T02:02:46.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.961+0000 7fb9fa000640 1 -- 172.21.15.103:0/4088610642 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f402fc40 con 0x7fb9fc112a30 2024-09-16T02:02:46.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.961+0000 7fba04200640 1 -- 172.21.15.103:0/4088610642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9fc19f270 con 0x7fb9fc112a30 2024-09-16T02:02:46.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.961+0000 7fba04200640 1 -- 172.21.15.103:0/4088610642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9fc1ab520 con 0x7fb9fc112a30 2024-09-16T02:02:46.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.962+0000 7fba04200640 1 -- 172.21.15.103:0/4088610642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9fc077e60 con 0x7fb9fc112a30 2024-09-16T02:02:46.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.964+0000 7fb9fa000640 1 -- 172.21.15.103:0/4088610642 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb9f4004050 con 0x7fb9fc112a30 2024-09-16T02:02:46.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.965+0000 7fb9fa000640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb9e40a32c0 0x7fb9e40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:46.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.966+0000 7fb9fa000640 1 -- 172.21.15.103:0/4088610642 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fb9f40f0640 con 0x7fb9fc112a30 2024-09-16T02:02:46.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.966+0000 7fba01800640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb9e40a32c0 0x7fb9e40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:46.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.966+0000 7fba01800640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb9e40a32c0 0x7fb9e40a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009bd0 tx=0x7fb9f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:46.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:46.970+0000 7fb9fa000640 1 -- 172.21.15.103:0/4088610642 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9f40bd760 con 0x7fb9fc112a30 2024-09-16T02:02:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.148+0000 7fba04200640 1 -- 172.21.15.103:0/4088610642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9fc076b00 con 0x7fb9fc112a30 2024-09-16T02:02:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:02:47.152 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":72,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:02:44.578736+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:02:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.149+0000 7fb9fa000640 1 -- 172.21.15.103:0/4088610642 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18550 (secure 0 0 0) 0x7fb9fc076b00 con 0x7fb9fc112a30 2024-09-16T02:02:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 -- 172.21.15.103:0/4088610642 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb9e40a32c0 msgr2=0x7fb9e40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb9e40a32c0 0x7fb9e40a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009bd0 tx=0x7fb9f0005f90 comp rx=0 tx=0).stop 2024-09-16T02:02:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 -- 172.21.15.103:0/4088610642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9fc112a30 msgr2=0x7fb9fc19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9fc112a30 0x7fb9fc19e840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9f402ade0 tx=0x7fb9f40319f0 comp rx=0 tx=0).stop 2024-09-16T02:02:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 -- 172.21.15.103:0/4088610642 shutdown_connections 2024-09-16T02:02:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb9e40a32c0 0x7fb9e40a5780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9fc112a30 0x7fb9fc19e840 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9fc077040 0x7fb9fc1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 --2- 172.21.15.103:0/4088610642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9fc076720 0x7fb9fc1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 -- 172.21.15.103:0/4088610642 >> 172.21.15.103:0/4088610642 conn(0x7fb9fc0ffa90 msgr2=0x7fb9fc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 -- 172.21.15.103:0/4088610642 shutdown_connections 2024-09-16T02:02:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:47.152+0000 7fba04200640 1 -- 172.21.15.103:0/4088610642 wait complete. 2024-09-16T02:02:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:47 smithi103 ceph-mon[30398]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:47.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:47 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/4088610642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:47.895 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T01:58:50.953500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T02:02:47.895 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T02:02:48.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:47 smithi172 ceph-mon[35340]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:48.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:47 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/4088610642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:48.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:47 smithi136 ceph-mon[35323]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:48.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:47 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/4088610642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:02:48.269 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:48.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.801+0000 7f0c5da00640 1 -- 172.21.15.103:0/2801524709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c58108f60 msgr2=0x7f0c58110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.801+0000 7f0c5da00640 1 --2- 172.21.15.103:0/2801524709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c58108f60 0x7f0c58110ae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009ca0 tx=0x7f0c5002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.801+0000 7f0c5da00640 1 -- 172.21.15.103:0/2801524709 shutdown_connections 2024-09-16T02:02:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.801+0000 7f0c5da00640 1 --2- 172.21.15.103:0/2801524709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c58108f60 0x7f0c58110ae0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.801+0000 7f0c5da00640 1 --2- 172.21.15.103:0/2801524709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581047e0 0x7f0c58108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.801+0000 7f0c5da00640 1 --2- 172.21.15.103:0/2801524709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58103e30 0x7f0c58104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.801+0000 7f0c5da00640 1 -- 172.21.15.103:0/2801524709 >> 172.21.15.103:0/2801524709 conn(0x7f0c580ff8d0 msgr2=0x7f0c58101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.801+0000 7f0c5da00640 1 -- 172.21.15.103:0/2801524709 shutdown_connections 2024-09-16T02:02:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.801+0000 7f0c5da00640 1 -- 172.21.15.103:0/2801524709 wait complete. 2024-09-16T02:02:48.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c5da00640 1 Processor -- start 2024-09-16T02:02:48.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c5da00640 1 -- start start 2024-09-16T02:02:48.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c5da00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c58103e30 0x7f0c5819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:48.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c5da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581047e0 0x7f0c5819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:48.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c5da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58108f60 0x7f0c5819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:48.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c5da00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c58114010 con 0x7f0c58103e30 2024-09-16T02:02:48.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c5da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0c58113e90 con 0x7f0c581047e0 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c5da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c58114190 con 0x7f0c58108f60 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c57e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c58103e30 0x7f0c5819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c57e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c58103e30 0x7f0c5819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52980/0 (socket says 172.21.15.103:52980) 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.803+0000 7f0c57e00640 1 -- 172.21.15.103:0/2297019732 learned_addr learned my addr 172.21.15.103:0/2297019732 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c5d000640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58108f60 0x7f0c5819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c57400640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581047e0 0x7f0c5819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c57e00640 1 -- 172.21.15.103:0/2297019732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58108f60 msgr2=0x7f0c5819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c57e00640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58108f60 0x7f0c5819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c57e00640 1 -- 172.21.15.103:0/2297019732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581047e0 msgr2=0x7f0c5819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c57e00640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581047e0 0x7f0c5819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c57e00640 1 -- 172.21.15.103:0/2297019732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5819f380 con 0x7f0c58103e30 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c57e00640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c58103e30 0x7f0c5819a3d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800e9f0 tx=0x7f0c4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c54c00640 1 -- 172.21.15.103:0/2297019732 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4800cd80 con 0x7f0c58103e30 2024-09-16T02:02:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c54c00640 1 -- 172.21.15.103:0/2297019732 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c48010910 con 0x7f0c58103e30 2024-09-16T02:02:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.804+0000 7f0c54c00640 1 -- 172.21.15.103:0/2297019732 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48018990 con 0x7f0c58103e30 2024-09-16T02:02:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.805+0000 7f0c5da00640 1 -- 172.21.15.103:0/2297019732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5819f670 con 0x7f0c58103e30 2024-09-16T02:02:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.805+0000 7f0c5da00640 1 -- 172.21.15.103:0/2297019732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c581a6f50 con 0x7f0c58103e30 2024-09-16T02:02:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.806+0000 7f0c5da00640 1 -- 172.21.15.103:0/2297019732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c58104210 con 0x7f0c58103e30 2024-09-16T02:02:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.807+0000 7f0c54c00640 1 -- 172.21.15.103:0/2297019732 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0c48010ac0 con 0x7f0c58103e30 2024-09-16T02:02:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.809+0000 7f0c54c00640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0c3c0a32c0 0x7f0c3c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.809+0000 7f0c54c00640 1 -- 172.21.15.103:0/2297019732 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0c480cf780 con 0x7f0c58103e30 2024-09-16T02:02:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.809+0000 7f0c57400640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0c3c0a32c0 0x7f0c3c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.810+0000 7f0c57400640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0c3c0a32c0 0x7f0c3c0a5780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c5810a930 tx=0x7f0c44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.813+0000 7f0c54c00640 1 -- 172.21.15.103:0/2297019732 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c4809c250 con 0x7f0c58103e30 2024-09-16T02:02:48.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.987+0000 7f0c5da00640 1 -- 172.21.15.103:0/2297019732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c5802d7c0 con 0x7f0c58103e30 2024-09-16T02:02:48.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.988+0000 7f0c54c00640 1 -- 172.21.15.103:0/2297019732 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f0c480a0550 con 0x7f0c58103e30 2024-09-16T02:02:48.994 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-16T02:02:48.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.990+0000 7f0c5da00640 1 -- 172.21.15.103:0/2297019732 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0c3c0a32c0 msgr2=0x7f0c3c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:48.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.990+0000 7f0c5da00640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0c3c0a32c0 0x7f0c3c0a5780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c5810a930 tx=0x7f0c44005f90 comp rx=0 tx=0).stop 2024-09-16T02:02:48.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.990+0000 7f0c5da00640 1 -- 172.21.15.103:0/2297019732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c58103e30 msgr2=0x7f0c5819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:48.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.990+0000 7f0c5da00640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c58103e30 0x7f0c5819a3d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800e9f0 tx=0x7f0c4800eeb0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.990+0000 7f0c5da00640 1 -- 172.21.15.103:0/2297019732 shutdown_connections 2024-09-16T02:02:48.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.990+0000 7f0c5da00640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0c3c0a32c0 0x7f0c3c0a5780 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.990+0000 7f0c5da00640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58108f60 0x7f0c5819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.990+0000 7f0c5da00640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581047e0 0x7f0c5819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.990+0000 7f0c5da00640 1 --2- 172.21.15.103:0/2297019732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c58103e30 0x7f0c5819a3d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:48.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.990+0000 7f0c5da00640 1 -- 172.21.15.103:0/2297019732 >> 172.21.15.103:0/2297019732 conn(0x7f0c580ff8d0 msgr2=0x7f0c58100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:48.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.991+0000 7f0c5da00640 1 -- 172.21.15.103:0/2297019732 shutdown_connections 2024-09-16T02:02:48.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:48.991+0000 7f0c5da00640 1 -- 172.21.15.103:0/2297019732 wait complete. 2024-09-16T02:02:49.695 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:49 smithi103 ceph-mon[30398]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:49.696 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:49 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2297019732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:02:49.733 INFO:tasks.cephadm:Adding mds.a on smithi103 2024-09-16T02:02:49.734 INFO:tasks.cephadm:Adding mds.d on smithi103 2024-09-16T02:02:49.734 INFO:tasks.cephadm:Adding mds.g on smithi103 2024-09-16T02:02:49.734 INFO:tasks.cephadm:Adding mds.j on smithi103 2024-09-16T02:02:49.734 INFO:tasks.cephadm:Adding mds.b on smithi136 2024-09-16T02:02:49.734 INFO:tasks.cephadm:Adding mds.e on smithi136 2024-09-16T02:02:49.734 INFO:tasks.cephadm:Adding mds.h on smithi136 2024-09-16T02:02:49.734 INFO:tasks.cephadm:Adding mds.k on smithi136 2024-09-16T02:02:49.734 INFO:tasks.cephadm:Adding mds.c on smithi172 2024-09-16T02:02:49.734 INFO:tasks.cephadm:Adding mds.f on smithi172 2024-09-16T02:02:49.735 INFO:tasks.cephadm:Adding mds.i on smithi172 2024-09-16T02:02:49.735 INFO:tasks.cephadm:Adding mds.l on smithi172 2024-09-16T02:02:49.735 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l' 2024-09-16T02:02:49.994 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:02:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:49 smithi136 ceph-mon[35323]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:49 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2297019732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:02:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:49 smithi172 ceph-mon[35340]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:49 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2297019732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:02:50.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.460+0000 7f9502820640 1 -- 172.21.15.172:0/3552471916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94fc101d10 msgr2=0x7f94fc102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.460+0000 7f9502820640 1 --2- 172.21.15.172:0/3552471916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94fc101d10 0x7f94fc102190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009900 tx=0x7f94f002a410 comp rx=0 tx=0).stop 2024-09-16T02:02:50.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.460+0000 7f9502820640 1 -- 172.21.15.172:0/3552471916 shutdown_connections 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.460+0000 7f9502820640 1 --2- 172.21.15.172:0/3552471916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94fc1026d0 0x7f94fc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.460+0000 7f9502820640 1 --2- 172.21.15.172:0/3552471916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94fc101d10 0x7f94fc102190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.460+0000 7f9502820640 1 --2- 172.21.15.172:0/3552471916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94fc100b10 0x7f94fc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.460+0000 7f9502820640 1 -- 172.21.15.172:0/3552471916 >> 172.21.15.172:0/3552471916 conn(0x7f94fc0fc2a0 msgr2=0x7f94fc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.460+0000 7f9502820640 1 -- 172.21.15.172:0/3552471916 shutdown_connections 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.460+0000 7f9502820640 1 -- 172.21.15.172:0/3552471916 wait complete. 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f9502820640 1 Processor -- start 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f9502820640 1 -- start start 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f9502820640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94fc100b10 0x7f94fc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f9502820640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94fc101d10 0x7f94fc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f9502820640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94fc1026d0 0x7f94fc19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f9502820640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc10d920 con 0x7f94fc101d10 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f9502820640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc10d7a0 con 0x7f94fc1026d0 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f9502820640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc10daa0 con 0x7f94fc100b10 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f94fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94fc100b10 0x7f94fc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f94fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94fc100b10 0x7f94fc197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46762/0 (socket says 172.21.15.172:46762) 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.461+0000 7f94fbfff640 1 -- 172.21.15.172:0/2886018702 learned_addr learned my addr 172.21.15.172:0/2886018702 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f9500dbd640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94fc1026d0 0x7f94fc19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f94fbfff640 1 -- 172.21.15.172:0/2886018702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94fc1026d0 msgr2=0x7f94fc19f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f94fbfff640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94fc1026d0 0x7f94fc19f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f94fbfff640 1 -- 172.21.15.172:0/2886018702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94fc101d10 msgr2=0x7f94fc1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f94fbfff640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94fc101d10 0x7f94fc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f94fbfff640 1 -- 172.21.15.172:0/2886018702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc19fbd0 con 0x7f94fc100b10 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f94fbfff640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94fc100b10 0x7f94fc197fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00d950 tx=0x7f94ec00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f94f97fa640 1 -- 172.21.15.172:0/2886018702 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec017070 con 0x7f94fc100b10 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f94f97fa640 1 -- 172.21.15.172:0/2886018702 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ec009c30 con 0x7f94fc100b10 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f9502820640 1 -- 172.21.15.172:0/2886018702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94fc19fec0 con 0x7f94fc100b10 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f94f97fa640 1 -- 172.21.15.172:0/2886018702 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec00f600 con 0x7f94fc100b10 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.462+0000 7f9502820640 1 -- 172.21.15.172:0/2886018702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94fc1a03a0 con 0x7f94fc100b10 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.463+0000 7f9502820640 1 -- 172.21.15.172:0/2886018702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94fc107ee0 con 0x7f94fc100b10 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.464+0000 7f94f97fa640 1 -- 172.21.15.172:0/2886018702 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f94ec01a070 con 0x7f94fc100b10 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.465+0000 7f94f97fa640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f94e40a3270 0x7f94e40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.465+0000 7f94f97fa640 1 -- 172.21.15.172:0/2886018702 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f94ec0cef70 con 0x7f94fc100b10 2024-09-16T02:02:50.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.465+0000 7f94fb7fe640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f94e40a3270 0x7f94e40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:50.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.466+0000 7f94fb7fe640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f94e40a3270 0x7f94e40a5730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f94f00096c0 tx=0x7f94f0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:50.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.469+0000 7f94f97fa640 1 -- 172.21.15.172:0/2886018702 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94ec09c130 con 0x7f94fc100b10 2024-09-16T02:02:50.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.646+0000 7f9502820640 1 -- 172.21.15.172:0/2886018702 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}) -- 0x7f94fc0627f0 con 0x7f94e40a3270 2024-09-16T02:02:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.656+0000 7f94f97fa640 1 -- 172.21.15.172:0/2886018702 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f94fc0627f0 con 0x7f94e40a3270 2024-09-16T02:02:50.659 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mds.all update... 2024-09-16T02:02:50.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.658+0000 7f9502820640 1 -- 172.21.15.172:0/2886018702 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f94e40a3270 msgr2=0x7f94e40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.658+0000 7f9502820640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f94e40a3270 0x7f94e40a5730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f94f00096c0 tx=0x7f94f0031400 comp rx=0 tx=0).stop 2024-09-16T02:02:50.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.658+0000 7f9502820640 1 -- 172.21.15.172:0/2886018702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94fc100b10 msgr2=0x7f94fc197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:50.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.658+0000 7f9502820640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94fc100b10 0x7f94fc197fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00d950 tx=0x7f94ec00de10 comp rx=0 tx=0).stop 2024-09-16T02:02:50.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.659+0000 7f9502820640 1 -- 172.21.15.172:0/2886018702 shutdown_connections 2024-09-16T02:02:50.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.659+0000 7f9502820640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f94e40a3270 0x7f94e40a5730 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.659+0000 7f9502820640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94fc1026d0 0x7f94fc19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.659+0000 7f9502820640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94fc101d10 0x7f94fc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.659+0000 7f9502820640 1 --2- 172.21.15.172:0/2886018702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94fc100b10 0x7f94fc197fa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:50.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.659+0000 7f9502820640 1 -- 172.21.15.172:0/2886018702 >> 172.21.15.172:0/2886018702 conn(0x7f94fc0fc2a0 msgr2=0x7f94fc1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:50.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.659+0000 7f9502820640 1 -- 172.21.15.172:0/2886018702 shutdown_connections 2024-09-16T02:02:50.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-16T02:02:50.659+0000 7f9502820640 1 -- 172.21.15.172:0/2886018702 wait complete. 2024-09-16T02:02:51.380 DEBUG:teuthology.orchestra.run.smithi103:mds.a> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.a.service 2024-09-16T02:02:51.385 DEBUG:teuthology.orchestra.run.smithi103:mds.d> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.d.service 2024-09-16T02:02:51.429 DEBUG:teuthology.orchestra.run.smithi103:mds.g> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.g.service 2024-09-16T02:02:51.475 DEBUG:teuthology.orchestra.run.smithi103:mds.j> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.j.service 2024-09-16T02:02:51.521 DEBUG:teuthology.orchestra.run.smithi136:mds.b> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.b.service 2024-09-16T02:02:51.524 DEBUG:teuthology.orchestra.run.smithi136:mds.e> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.e.service 2024-09-16T02:02:51.570 DEBUG:teuthology.orchestra.run.smithi136:mds.h> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.h.service 2024-09-16T02:02:51.613 DEBUG:teuthology.orchestra.run.smithi136:mds.k> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.k.service 2024-09-16T02:02:51.638 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:51 smithi172 ceph-mon[35340]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:51.638 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:51.638 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:51.638 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:51.639 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:51.639 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:51.639 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:02:51.639 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:02:51.639 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:51.656 DEBUG:teuthology.orchestra.run.smithi172:mds.c> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.c.service 2024-09-16T02:02:51.659 DEBUG:teuthology.orchestra.run.smithi172:mds.f> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.f.service 2024-09-16T02:02:51.702 DEBUG:teuthology.orchestra.run.smithi172:mds.i> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.i.service 2024-09-16T02:02:51.746 DEBUG:teuthology.orchestra.run.smithi172:mds.l> sudo journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.l.service 2024-09-16T02:02:51.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:51 smithi103 ceph-mon[30398]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:51.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:51.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:02:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:02:51.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:51.789 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-16T02:02:51.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:02:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:51 smithi136 ceph-mon[35323]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:02:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:02:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:52.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:02:52.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:02:52.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:52.154 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:52.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.617+0000 7fb6c7000640 1 -- 172.21.15.103:0/931991018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c0101de0 msgr2=0x7fb6c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:52.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.617+0000 7fb6c7000640 1 --2- 172.21.15.103:0/931991018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c0101de0 0x7fb6c01021c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009900 tx=0x7fb6b002a410 comp rx=0 tx=0).stop 2024-09-16T02:02:52.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.617+0000 7fb6c7000640 1 -- 172.21.15.103:0/931991018 shutdown_connections 2024-09-16T02:02:52.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.617+0000 7fb6c7000640 1 --2- 172.21.15.103:0/931991018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6c0106f10 0x7fb6c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:52.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.617+0000 7fb6c7000640 1 --2- 172.21.15.103:0/931991018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6c0102790 0x7fb6c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:52.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.617+0000 7fb6c7000640 1 --2- 172.21.15.103:0/931991018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c0101de0 0x7fb6c01021c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:52.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.617+0000 7fb6c7000640 1 -- 172.21.15.103:0/931991018 >> 172.21.15.103:0/931991018 conn(0x7fb6c00fd8c0 msgr2=0x7fb6c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:52.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.617+0000 7fb6c7000640 1 -- 172.21.15.103:0/931991018 shutdown_connections 2024-09-16T02:02:52.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.617+0000 7fb6c7000640 1 -- 172.21.15.103:0/931991018 wait complete. 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.618+0000 7fb6c7000640 1 Processor -- start 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.618+0000 7fb6c7000640 1 -- start start 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6c7000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c0101de0 0x7fb6c019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6c7000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6c0102790 0x7fb6c019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6c7000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6c0106f10 0x7fb6c019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6c7000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c0111fc0 con 0x7fb6c0101de0 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6c7000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c0111e40 con 0x7fb6c0102790 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6c7000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c0112140 con 0x7fb6c0106f10 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c0101de0 0x7fb6c019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c0101de0 0x7fb6c019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52992/0 (socket says 172.21.15.103:52992) 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6bfe00640 1 -- 172.21.15.103:0/1322630506 learned_addr learned my addr 172.21.15.103:0/1322630506 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:02:52.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6bf400640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6c0102790 0x7fb6c019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:52.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.619+0000 7fb6c5000640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6c0106f10 0x7fb6c019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:52.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6bfe00640 1 -- 172.21.15.103:0/1322630506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6c0106f10 msgr2=0x7fb6c019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:52.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6bfe00640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6c0106f10 0x7fb6c019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:52.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6bfe00640 1 -- 172.21.15.103:0/1322630506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6c0102790 msgr2=0x7fb6c019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:52.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6bfe00640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6c0102790 0x7fb6c019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:52.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6bfe00640 1 -- 172.21.15.103:0/1322630506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c019f5e0 con 0x7fb6c0101de0 2024-09-16T02:02:52.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6bfe00640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c0101de0 0x7fb6c019a630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0004770 tx=0x7fb6b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:52.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6bcc00640 1 -- 172.21.15.103:0/1322630506 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b003f070 con 0x7fb6c0101de0 2024-09-16T02:02:52.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6bcc00640 1 -- 172.21.15.103:0/1322630506 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6b0031ca0 con 0x7fb6c0101de0 2024-09-16T02:02:52.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6c7000640 1 -- 172.21.15.103:0/1322630506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c019f870 con 0x7fb6c0101de0 2024-09-16T02:02:52.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6bcc00640 1 -- 172.21.15.103:0/1322630506 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0035030 con 0x7fb6c0101de0 2024-09-16T02:02:52.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.620+0000 7fb6c7000640 1 -- 172.21.15.103:0/1322630506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c01a71b0 con 0x7fb6c0101de0 2024-09-16T02:02:52.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.622+0000 7fb6c7000640 1 -- 172.21.15.103:0/1322630506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c01021c0 con 0x7fb6c0101de0 2024-09-16T02:02:52.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.623+0000 7fb6bcc00640 1 -- 172.21.15.103:0/1322630506 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb6b0042070 con 0x7fb6c0101de0 2024-09-16T02:02:52.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.624+0000 7fb6bcc00640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb6a40a32c0 0x7fb6a40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:52.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.625+0000 7fb6bcc00640 1 -- 172.21.15.103:0/1322630506 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fb6b00efae0 con 0x7fb6c0101de0 2024-09-16T02:02:52.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.625+0000 7fb6bf400640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb6a40a32c0 0x7fb6a40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:52.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.625+0000 7fb6bf400640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb6a40a32c0 0x7fb6a40a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0076480 tx=0x7fb6ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:52.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.628+0000 7fb6bcc00640 1 -- 172.21.15.103:0/1322630506 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb6b00bc5b0 con 0x7fb6c0101de0 2024-09-16T02:02:52.776 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:52 smithi172 ceph-mon[35340]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:52.776 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:52 smithi172 ceph-mon[35340]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-09-16T02:02:52.776 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:52 smithi172 ceph-mon[35340]: Deploying daemon mds.l on smithi172 2024-09-16T02:02:52.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.842+0000 7fb6c7000640 1 -- 172.21.15.103:0/1322630506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6c01a7380 con 0x7fb6c0101de0 2024-09-16T02:02:52.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:52 smithi103 ceph-mon[30398]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:52.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:52 smithi103 ceph-mon[30398]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-09-16T02:02:52.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:52 smithi103 ceph-mon[30398]: Deploying daemon mds.l on smithi172 2024-09-16T02:02:52.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.843+0000 7fb6bcc00640 1 -- 172.21.15.103:0/1322630506 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fb6b00c08b0 con 0x7fb6c0101de0 2024-09-16T02:02:52.846 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:02:52.846 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":1,"btime":"2024-09-16T01:55:37:666788+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-16T02:02:52.846 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 1 2024-09-16T02:02:52.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.845+0000 7fb6c7000640 1 -- 172.21.15.103:0/1322630506 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb6a40a32c0 msgr2=0x7fb6a40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:52.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.845+0000 7fb6c7000640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb6a40a32c0 0x7fb6a40a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0076480 tx=0x7fb6ac005f90 comp rx=0 tx=0).stop 2024-09-16T02:02:52.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.845+0000 7fb6c7000640 1 -- 172.21.15.103:0/1322630506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c0101de0 msgr2=0x7fb6c019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:52.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.845+0000 7fb6c7000640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c0101de0 0x7fb6c019a630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0004770 tx=0x7fb6b0003c10 comp rx=0 tx=0).stop 2024-09-16T02:02:52.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.846+0000 7fb6c7000640 1 -- 172.21.15.103:0/1322630506 shutdown_connections 2024-09-16T02:02:52.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.846+0000 7fb6c7000640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb6a40a32c0 0x7fb6a40a5780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:52.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.846+0000 7fb6c7000640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6c0106f10 0x7fb6c019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:52.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.846+0000 7fb6c7000640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6c0102790 0x7fb6c019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:52.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.846+0000 7fb6c7000640 1 --2- 172.21.15.103:0/1322630506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c0101de0 0x7fb6c019a630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:52.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.846+0000 7fb6c7000640 1 -- 172.21.15.103:0/1322630506 >> 172.21.15.103:0/1322630506 conn(0x7fb6c00fd8c0 msgr2=0x7fb6c00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:52.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.846+0000 7fb6c7000640 1 -- 172.21.15.103:0/1322630506 shutdown_connections 2024-09-16T02:02:52.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:52.846+0000 7fb6c7000640 1 -- 172.21.15.103:0/1322630506 wait complete. 2024-09-16T02:02:53.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:52 smithi136 ceph-mon[35323]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:02:53.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:52 smithi136 ceph-mon[35323]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-09-16T02:02:53.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:52 smithi136 ceph-mon[35323]: Deploying daemon mds.l on smithi172 2024-09-16T02:02:53.045 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:02:52 smithi172 systemd[1]: Starting Ceph mds.l for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:02:53.438 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:02:53 smithi172 podman[65216]: 2024-09-16 02:02:53.073115394 +0000 UTC m=+0.015751844 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:02:53.438 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:02:53 smithi172 podman[65216]: 2024-09-16 02:02:53.191898896 +0000 UTC m=+0.134535346 container create 1f7e1640479fe8671d97b6a3e22480d77e17c1028c9a5f40fc702661072420da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T02:02:53.438 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:02:53 smithi172 podman[65216]: 2024-09-16 02:02:53.410956723 +0000 UTC m=+0.353593173 container init 1f7e1640479fe8671d97b6a3e22480d77e17c1028c9a5f40fc702661072420da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e) 2024-09-16T02:02:53.438 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:02:53 smithi172 podman[65216]: 2024-09-16 02:02:53.413827905 +0000 UTC m=+0.356464364 container start 1f7e1640479fe8671d97b6a3e22480d77e17c1028c9a5f40fc702661072420da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T02:02:53.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1322630506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.172:6832/1362058923,v1:172.21.15.172:6833/1362058923] up:boot 2024-09-16T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: fsmap 1 up:standby 2024-09-16T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:53 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1322630506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.172:6832/1362058923,v1:172.21.15.172:6833/1362058923] up:boot 2024-09-16T02:02:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: fsmap 1 up:standby 2024-09-16T02:02:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T02:02:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:02:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:02:53.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:53.796 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-mds[65236]: -- 172.21.15.172:0/562328557 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603f1d07860 con 0x5603f1ce5400 2024-09-16T02:02:53.796 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:02:53 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-l[65232]: starting mds.l at 2024-09-16T02:02:53.796 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:02:53 smithi172 bash[65216]: 1f7e1640479fe8671d97b6a3e22480d77e17c1028c9a5f40fc702661072420da 2024-09-16T02:02:53.796 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:02:53 smithi172 systemd[1]: Started Ceph mds.l for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:02:54.018 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:54.018 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1322630506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:54.018 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.172:6832/1362058923,v1:172.21.15.172:6833/1362058923] up:boot 2024-09-16T02:02:54.018 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: fsmap 1 up:standby 2024-09-16T02:02:54.018 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T02:02:54.018 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:54.019 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:54.019 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:54.019 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:02:54.019 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:02:54.019 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:53 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:54.898 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:54 smithi136 ceph-mon[35323]: Deploying daemon mds.k on smithi136 2024-09-16T02:02:55.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:54 smithi103 ceph-mon[30398]: Deploying daemon mds.k on smithi136 2024-09-16T02:02:55.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:54 smithi172 ceph-mon[35340]: Deploying daemon mds.k on smithi136 2024-09-16T02:02:55.523 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:02:55 smithi136 systemd[1]: Starting Ceph mds.k for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:02:55.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:02:55.870 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:55 smithi136 ceph-mon[35323]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:55.870 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:02:55 smithi136 podman[65133]: 2024-09-16 02:02:55.466814536 +0000 UTC m=+0.015907539 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:02:55.871 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:02:55 smithi136 podman[65133]: 2024-09-16 02:02:55.592312333 +0000 UTC m=+0.141405326 container create 3c88d7cf0f4b03210a3f3abbdca8df55a6db9ade1f08ff5913706e77ebd1d6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-k, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T02:02:55.871 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:02:55 smithi136 podman[65133]: 2024-09-16 02:02:55.786407329 +0000 UTC m=+0.335500320 container init 3c88d7cf0f4b03210a3f3abbdca8df55a6db9ade1f08ff5913706e77ebd1d6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-k, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T02:02:55.871 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:02:55 smithi136 podman[65133]: 2024-09-16 02:02:55.789257279 +0000 UTC m=+0.338350270 container start 3c88d7cf0f4b03210a3f3abbdca8df55a6db9ade1f08ff5913706e77ebd1d6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T02:02:55.871 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:02:55 smithi136 ceph-mds[65152]: -- 172.21.15.136:0/1662197182 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef0a969860 con 0x55ef0a947400 2024-09-16T02:02:55.871 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:02:55 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-k[65148]: starting mds.k at 2024-09-16T02:02:55.871 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:02:55 smithi136 bash[65133]: 3c88d7cf0f4b03210a3f3abbdca8df55a6db9ade1f08ff5913706e77ebd1d6b3 2024-09-16T02:02:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:02:56.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:55 smithi103 ceph-mon[30398]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:56.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:55 smithi172 ceph-mon[35340]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:56.273 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:02:55 smithi136 systemd[1]: Started Ceph mds.k for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:02:56.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.392+0000 7f9be7200640 1 -- 172.21.15.103:0/431167495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9be8104800 msgr2=0x7f9be8108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:56.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.392+0000 7f9be7200640 1 --2- 172.21.15.103:0/431167495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9be8104800 0x7f9be8108960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009900 tx=0x7f9bd002a3f0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.392+0000 7f9be7200640 1 -- 172.21.15.103:0/431167495 shutdown_connections 2024-09-16T02:02:56.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.392+0000 7f9be7200640 1 --2- 172.21.15.103:0/431167495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be8109090 0x7f9be8110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.392+0000 7f9be7200640 1 --2- 172.21.15.103:0/431167495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9be8104800 0x7f9be8108960 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.392+0000 7f9be7200640 1 --2- 172.21.15.103:0/431167495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8103e50 0x7f9be8104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.392+0000 7f9be7200640 1 -- 172.21.15.103:0/431167495 >> 172.21.15.103:0/431167495 conn(0x7f9be80ff8b0 msgr2=0x7f9be8101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:56.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.392+0000 7f9be7200640 1 -- 172.21.15.103:0/431167495 shutdown_connections 2024-09-16T02:02:56.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.392+0000 7f9be7200640 1 -- 172.21.15.103:0/431167495 wait complete. 2024-09-16T02:02:56.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.393+0000 7f9be7200640 1 Processor -- start 2024-09-16T02:02:56.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.393+0000 7f9be7200640 1 -- start start 2024-09-16T02:02:56.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8103e50 0x7f9be819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:56.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be8104800 0x7f9be819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:56.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be7200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9be8109090 0x7f9be819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:56.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81140b0 con 0x7f9be8104800 2024-09-16T02:02:56.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be7200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9be8113f30 con 0x7f9be8109090 2024-09-16T02:02:56.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9be8114230 con 0x7f9be8103e50 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8103e50 0x7f9be819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9bdfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be8104800 0x7f9be819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8103e50 0x7f9be819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:43566/0 (socket says 172.21.15.103:43566) 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be5e00640 1 -- 172.21.15.103:0/432661626 learned_addr learned my addr 172.21.15.103:0/432661626 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be6800640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9be8109090 0x7f9be819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be5e00640 1 -- 172.21.15.103:0/432661626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9be8109090 msgr2=0x7f9be819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be5e00640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9be8109090 0x7f9be819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be5e00640 1 -- 172.21.15.103:0/432661626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be8104800 msgr2=0x7f9be819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be5e00640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be8104800 0x7f9be819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.394+0000 7f9be5e00640 1 -- 172.21.15.103:0/432661626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be819f5e0 con 0x7f9be8103e50 2024-09-16T02:02:56.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.395+0000 7f9be5e00640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8103e50 0x7f9be819a630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800e9f0 tx=0x7f9bd800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:56.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.395+0000 7f9bdea00640 1 -- 172.21.15.103:0/432661626 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800ccc0 con 0x7f9be8103e50 2024-09-16T02:02:56.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.395+0000 7f9bdea00640 1 -- 172.21.15.103:0/432661626 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bd800ce60 con 0x7f9be8103e50 2024-09-16T02:02:56.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.395+0000 7f9be7200640 1 -- 172.21.15.103:0/432661626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be819f8d0 con 0x7f9be8103e50 2024-09-16T02:02:56.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.395+0000 7f9bdea00640 1 -- 172.21.15.103:0/432661626 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8010690 con 0x7f9be8103e50 2024-09-16T02:02:56.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.395+0000 7f9be7200640 1 -- 172.21.15.103:0/432661626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be81a71b0 con 0x7f9be8103e50 2024-09-16T02:02:56.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.396+0000 7f9be7200640 1 -- 172.21.15.103:0/432661626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be8105330 con 0x7f9be8103e50 2024-09-16T02:02:56.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.400+0000 7f9bdea00640 1 -- 172.21.15.103:0/432661626 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9bd80046c0 con 0x7f9be8103e50 2024-09-16T02:02:56.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.401+0000 7f9bdea00640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9bcc0a3240 0x7f9bcc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:02:56.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.401+0000 7f9bdea00640 1 -- 172.21.15.103:0/432661626 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f9bd80cfbc0 con 0x7f9be8103e50 2024-09-16T02:02:56.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.401+0000 7f9bdfe00640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9bcc0a3240 0x7f9bcc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:02:56.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.402+0000 7f9bdea00640 1 -- 172.21.15.103:0/432661626 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9bd809cce0 con 0x7f9be8103e50 2024-09-16T02:02:56.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.402+0000 7f9bdfe00640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9bcc0a3240 0x7f9bcc0a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009230 tx=0x7f9bd00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:02:56.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.637+0000 7f9be7200640 1 -- 172.21.15.103:0/432661626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9be8104230 con 0x7f9be8103e50 2024-09-16T02:02:56.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.639+0000 7f9bdea00640 1 -- 172.21.15.103:0/432661626 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f9bd809c5e0 con 0x7f9be8103e50 2024-09-16T02:02:56.640 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:02:56.640 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":3,"btime":"2024-09-16T02:02:56:613048+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T02:02:56.641 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 3 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.641+0000 7f9be7200640 1 -- 172.21.15.103:0/432661626 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9bcc0a3240 msgr2=0x7f9bcc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.641+0000 7f9be7200640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9bcc0a3240 0x7f9bcc0a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009230 tx=0x7f9bd00058d0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.641+0000 7f9be7200640 1 -- 172.21.15.103:0/432661626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8103e50 msgr2=0x7f9be819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.641+0000 7f9be7200640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8103e50 0x7f9be819a630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800e9f0 tx=0x7f9bd800edf0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.642+0000 7f9be7200640 1 -- 172.21.15.103:0/432661626 shutdown_connections 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.642+0000 7f9be7200640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9bcc0a3240 0x7f9bcc0a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.642+0000 7f9be7200640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9be8109090 0x7f9be819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.642+0000 7f9be7200640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be8104800 0x7f9be819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.642+0000 7f9be7200640 1 --2- 172.21.15.103:0/432661626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8103e50 0x7f9be819a630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.642+0000 7f9be7200640 1 -- 172.21.15.103:0/432661626 >> 172.21.15.103:0/432661626 conn(0x7f9be80ff8b0 msgr2=0x7f9be8100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:02:56.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.642+0000 7f9be7200640 1 -- 172.21.15.103:0/432661626 shutdown_connections 2024-09-16T02:02:56.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:02:56.642+0000 7f9be7200640 1 -- 172.21.15.103:0/432661626 wait complete. 2024-09-16T02:02:57.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:57.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:57.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:57.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:02:57.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:02:57.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:57.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: Deploying daemon mds.j on smithi103 2024-09-16T02:02:57.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:57.235 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.136:6832/3604147096,v1:172.21.15.136:6833/3604147096] up:boot 2024-09-16T02:02:57.235 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: fsmap 2 up:standby 2024-09-16T02:02:57.235 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T02:02:57.235 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:56 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/432661626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:57.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:57.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:57.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:57.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:02:57.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:02:57.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:57.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: Deploying daemon mds.j on smithi103 2024-09-16T02:02:57.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:57.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.136:6832/3604147096,v1:172.21.15.136:6833/3604147096] up:boot 2024-09-16T02:02:57.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: fsmap 2 up:standby 2024-09-16T02:02:57.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T02:02:57.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:56 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/432661626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:57.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:57.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:57.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:02:57.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:02:57.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:02:57.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:02:57.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: Deploying daemon mds.j on smithi103 2024-09-16T02:02:57.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:57.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.136:6832/3604147096,v1:172.21.15.136:6833/3604147096] up:boot 2024-09-16T02:02:57.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: fsmap 2 up:standby 2024-09-16T02:02:57.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T02:02:57.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:56 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/432661626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:02:59.255 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:02:58 smithi103 systemd[1]: Starting Ceph mds.j for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:02:59.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:02:59.548 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:02:59 smithi103 ceph-mon[30398]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:59.549 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:02:59 smithi103 podman[70091]: 2024-09-16 02:02:59.193267786 +0000 UTC m=+0.027404459 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:02:59.549 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:02:59 smithi103 podman[70091]: 2024-09-16 02:02:59.29279029 +0000 UTC m=+0.126927010 container create c975b03dbfa12486c55935749c0043750ea8484da7ddf3d496a105ac0f3af60a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:02:59.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:02:59 smithi136 ceph-mon[35323]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:02:59 smithi172 ceph-mon[35340]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:02:59.914 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:00.005 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:02:59 smithi103 podman[70091]: 2024-09-16 02:02:59.547747855 +0000 UTC m=+0.381884544 container init c975b03dbfa12486c55935749c0043750ea8484da7ddf3d496a105ac0f3af60a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-16T02:03:00.005 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:02:59 smithi103 podman[70091]: 2024-09-16 02:02:59.555226826 +0000 UTC m=+0.389363495 container start c975b03dbfa12486c55935749c0043750ea8484da7ddf3d496a105ac0f3af60a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T02:03:00.005 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:02:59 smithi103 ceph-mds[70128]: -- 172.21.15.103:0/1852114164 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef04c67860 con 0x55ef04c45000 2024-09-16T02:03:00.005 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:02:59 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-j[70107]: starting mds.j at 2024-09-16T02:03:00.005 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:02:59 smithi103 bash[70091]: c975b03dbfa12486c55935749c0043750ea8484da7ddf3d496a105ac0f3af60a 2024-09-16T02:03:00.005 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:02:59 smithi103 systemd[1]: Started Ceph mds.j for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:03:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.420+0000 7f33d8e00640 1 -- 172.21.15.103:0/3903581370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d4101de0 msgr2=0x7f33d41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.420+0000 7f33d8e00640 1 --2- 172.21.15.103:0/3903581370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d4101de0 0x7f33d41021c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009900 tx=0x7f33c402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:03:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.420+0000 7f33d8e00640 1 -- 172.21.15.103:0/3903581370 shutdown_connections 2024-09-16T02:03:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.420+0000 7f33d8e00640 1 --2- 172.21.15.103:0/3903581370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33d4106f10 0x7f33d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.420+0000 7f33d8e00640 1 --2- 172.21.15.103:0/3903581370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d4102790 0x7f33d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.420+0000 7f33d8e00640 1 --2- 172.21.15.103:0/3903581370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d4101de0 0x7f33d41021c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.420+0000 7f33d8e00640 1 -- 172.21.15.103:0/3903581370 >> 172.21.15.103:0/3903581370 conn(0x7f33d40fd8c0 msgr2=0x7f33d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.421+0000 7f33d8e00640 1 -- 172.21.15.103:0/3903581370 shutdown_connections 2024-09-16T02:03:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.421+0000 7f33d8e00640 1 -- 172.21.15.103:0/3903581370 wait complete. 2024-09-16T02:03:00.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.422+0000 7f33d8e00640 1 Processor -- start 2024-09-16T02:03:00.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.422+0000 7f33d8e00640 1 -- start start 2024-09-16T02:03:00.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.422+0000 7f33d8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33d4101de0 0x7f33d40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.422+0000 7f33d8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d4102790 0x7f33d4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.422+0000 7f33d8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d4106f10 0x7f33d4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.422+0000 7f33d8e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33d4111fc0 con 0x7f33d4106f10 2024-09-16T02:03:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.422+0000 7f33d8e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33d4111e40 con 0x7f33d4101de0 2024-09-16T02:03:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.422+0000 7f33d8e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33d4112140 con 0x7f33d4102790 2024-09-16T02:03:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33d4101de0 0x7f33d40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d4102790 0x7f33d4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33d4101de0 0x7f33d40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49782/0 (socket says 172.21.15.103:49782) 2024-09-16T02:03:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d1e00640 1 -- 172.21.15.103:0/3278269621 learned_addr learned my addr 172.21.15.103:0/3278269621 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d2800640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d4106f10 0x7f33d4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d1e00640 1 -- 172.21.15.103:0/3278269621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d4102790 msgr2=0x7f33d4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d1e00640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d4102790 0x7f33d4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d1e00640 1 -- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d4106f10 msgr2=0x7f33d4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d1e00640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d4106f10 0x7f33d4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d1e00640 1 -- 172.21.15.103:0/3278269621 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d41a6900 con 0x7f33d4101de0 2024-09-16T02:03:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.423+0000 7f33d2800640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d4106f10 0x7f33d4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.424+0000 7f33d1e00640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33d4101de0 0x7f33d40744f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f33c4004770 tx=0x7f33c4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.424+0000 7f33caa00640 1 -- 172.21.15.103:0/3278269621 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c403f070 con 0x7f33d4101de0 2024-09-16T02:03:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.424+0000 7f33caa00640 1 -- 172.21.15.103:0/3278269621 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c4031ca0 con 0x7f33d4101de0 2024-09-16T02:03:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.424+0000 7f33d8e00640 1 -- 172.21.15.103:0/3278269621 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d41a6b90 con 0x7f33d4101de0 2024-09-16T02:03:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.424+0000 7f33caa00640 1 -- 172.21.15.103:0/3278269621 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4035030 con 0x7f33d4101de0 2024-09-16T02:03:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.424+0000 7f33d8e00640 1 -- 172.21.15.103:0/3278269621 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d41a7070 con 0x7f33d4101de0 2024-09-16T02:03:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.425+0000 7f33d8e00640 1 -- 172.21.15.103:0/3278269621 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d41021c0 con 0x7f33d4101de0 2024-09-16T02:03:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.427+0000 7f33caa00640 1 -- 172.21.15.103:0/3278269621 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f33c4042070 con 0x7f33d4101de0 2024-09-16T02:03:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.428+0000 7f33caa00640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f33b80a3240 0x7f33b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.428+0000 7f33caa00640 1 -- 172.21.15.103:0/3278269621 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f33c40efb80 con 0x7f33d4101de0 2024-09-16T02:03:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.429+0000 7f33d1400640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f33b80a3240 0x7f33b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.429+0000 7f33d1400640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f33b80a3240 0x7f33b80a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f33d4075f80 tx=0x7f33c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.432+0000 7f33caa00640 1 -- 172.21.15.103:0/3278269621 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33c40bcd60 con 0x7f33d4101de0 2024-09-16T02:03:00.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.666+0000 7f33d8e00640 1 -- 172.21.15.103:0/3278269621 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33d410a0f0 con 0x7f33d4101de0 2024-09-16T02:03:00.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.667+0000 7f33caa00640 1 -- 172.21.15.103:0/3278269621 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f33c40bc660 con 0x7f33d4101de0 2024-09-16T02:03:00.668 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:00.668 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":4,"btime":"2024-09-16T02:03:00:498630+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T02:03:00.669 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 4 2024-09-16T02:03:00.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.669+0000 7f33d8e00640 1 -- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f33b80a3240 msgr2=0x7f33b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:00.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.669+0000 7f33d8e00640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f33b80a3240 0x7f33b80a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f33d4075f80 tx=0x7f33c0005f90 comp rx=0 tx=0).stop 2024-09-16T02:03:00.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.669+0000 7f33d8e00640 1 -- 172.21.15.103:0/3278269621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33d4101de0 msgr2=0x7f33d40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:00.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.669+0000 7f33d8e00640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33d4101de0 0x7f33d40744f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f33c4004770 tx=0x7f33c4003c10 comp rx=0 tx=0).stop 2024-09-16T02:03:00.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.669+0000 7f33d8e00640 1 -- 172.21.15.103:0/3278269621 shutdown_connections 2024-09-16T02:03:00.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.669+0000 7f33d8e00640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f33b80a3240 0x7f33b80a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:00.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.669+0000 7f33d8e00640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d4106f10 0x7f33d4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:00.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.669+0000 7f33d8e00640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d4102790 0x7f33d4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:00.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.669+0000 7f33d8e00640 1 --2- 172.21.15.103:0/3278269621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33d4101de0 0x7f33d40744f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:00.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.669+0000 7f33d8e00640 1 -- 172.21.15.103:0/3278269621 >> 172.21.15.103:0/3278269621 conn(0x7f33d40fd8c0 msgr2=0x7f33d40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:00.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.670+0000 7f33d8e00640 1 -- 172.21.15.103:0/3278269621 shutdown_connections 2024-09-16T02:03:00.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:00.670+0000 7f33d8e00640 1 -- 172.21.15.103:0/3278269621 wait complete. 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: Deploying daemon mds.d on smithi103 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.103:6834/2357019382,v1:172.21.15.103:6835/2357019382] up:boot 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: fsmap 3 up:standby 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T02:03:01.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:00 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3278269621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:01.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: Deploying daemon mds.d on smithi103 2024-09-16T02:03:01.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:01.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.103:6834/2357019382,v1:172.21.15.103:6835/2357019382] up:boot 2024-09-16T02:03:01.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: fsmap 3 up:standby 2024-09-16T02:03:01.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T02:03:01.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:00 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3278269621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:01.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:01.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:01.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:01.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:01.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:01.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:01.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: Deploying daemon mds.d on smithi103 2024-09-16T02:03:01.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:01.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.103:6834/2357019382,v1:172.21.15.103:6835/2357019382] up:boot 2024-09-16T02:03:01.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: fsmap 3 up:standby 2024-09-16T02:03:01.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T02:03:01.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:00 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3278269621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:03.163 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:03:02 smithi103 systemd[1]: Starting Ceph mds.d for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:03:03.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:03:03.427 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:03:03 smithi103 podman[70504]: 2024-09-16 02:03:03.065360046 +0000 UTC m=+0.027698463 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:03:03.428 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:03:03 smithi103 podman[70504]: 2024-09-16 02:03:03.182189951 +0000 UTC m=+0.144528365 container create 3a94b3dabb4f410d79d44d15334c497c2b580dc2426c436b3b11eae022f4e428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-16T02:03:03.428 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:03:03 smithi103 podman[70504]: 2024-09-16 02:03:03.426649409 +0000 UTC m=+0.388987826 container init 3a94b3dabb4f410d79d44d15334c497c2b580dc2426c436b3b11eae022f4e428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T02:03:03.738 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:03.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:03 smithi103 ceph-mon[30398]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:03.754 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:03:03 smithi103 podman[70504]: 2024-09-16 02:03:03.434586723 +0000 UTC m=+0.396925141 container start 3a94b3dabb4f410d79d44d15334c497c2b580dc2426c436b3b11eae022f4e428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T02:03:03.755 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:03:03 smithi103 ceph-mds[70541]: -- 172.21.15.103:0/3093445963 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f66e641c20 con 0x55f66e61e800 2024-09-16T02:03:03.755 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:03:03 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-d[70519]: starting mds.d at 2024-09-16T02:03:03.755 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:03:03 smithi103 bash[70504]: 3a94b3dabb4f410d79d44d15334c497c2b580dc2426c436b3b11eae022f4e428 2024-09-16T02:03:03.755 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:03:03 smithi103 systemd[1]: Started Ceph mds.d for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:03:03.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:03 smithi136 ceph-mon[35323]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:03.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:03 smithi172 ceph-mon[35340]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:04.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.266+0000 7f284a600640 1 -- 172.21.15.103:0/3538398271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2844074eb0 msgr2=0x7f2844112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:04.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.266+0000 7f284a600640 1 --2- 172.21.15.103:0/3538398271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2844074eb0 0x7f2844112650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f282c009900 tx=0x7f282c02a480 comp rx=0 tx=0).stop 2024-09-16T02:03:04.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.266+0000 7f284a600640 1 -- 172.21.15.103:0/3538398271 shutdown_connections 2024-09-16T02:03:04.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.266+0000 7f284a600640 1 --2- 172.21.15.103:0/3538398271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844112b90 0x7f2844114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:04.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.266+0000 7f284a600640 1 --2- 172.21.15.103:0/3538398271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2844074eb0 0x7f2844112650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:04.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.266+0000 7f284a600640 1 --2- 172.21.15.103:0/3538398271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2844074590 0x7f2844074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:04.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.266+0000 7f284a600640 1 -- 172.21.15.103:0/3538398271 >> 172.21.15.103:0/3538398271 conn(0x7f28440fd8c0 msgr2=0x7f28440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:04.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.266+0000 7f284a600640 1 -- 172.21.15.103:0/3538398271 shutdown_connections 2024-09-16T02:03:04.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.266+0000 7f284a600640 1 -- 172.21.15.103:0/3538398271 wait complete. 2024-09-16T02:03:04.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.267+0000 7f284a600640 1 Processor -- start 2024-09-16T02:03:04.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.267+0000 7f284a600640 1 -- start start 2024-09-16T02:03:04.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.267+0000 7f284a600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2844074590 0x7f28441a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:04.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.267+0000 7f284a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844074eb0 0x7f28441a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:04.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.267+0000 7f284a600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2844112b90 0x7f284419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:04.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.267+0000 7f284a600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28441185b0 con 0x7f2844112b90 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.267+0000 7f284a600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2844118430 con 0x7f2844074eb0 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.267+0000 7f284a600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2844118730 con 0x7f2844074590 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f283aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844074eb0 0x7f28441a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f283aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844074eb0 0x7f28441a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49824/0 (socket says 172.21.15.103:49824) 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f283aa00640 1 -- 172.21.15.103:0/1220760528 learned_addr learned my addr 172.21.15.103:0/1220760528 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f2843400640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2844074590 0x7f28441a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f2843e00640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2844112b90 0x7f284419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f283aa00640 1 -- 172.21.15.103:0/1220760528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2844074590 msgr2=0x7f28441a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f283aa00640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2844074590 0x7f28441a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f283aa00640 1 -- 172.21.15.103:0/1220760528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2844112b90 msgr2=0x7f284419e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f283aa00640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2844112b90 0x7f284419e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:04.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f283aa00640 1 -- 172.21.15.103:0/1220760528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284419f030 con 0x7f2844074eb0 2024-09-16T02:03:04.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f2843400640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2844074590 0x7f28441a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:04.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.268+0000 7f283aa00640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844074eb0 0x7f28441a4c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f282c004770 tx=0x7f282c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:04.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.269+0000 7f2840c00640 1 -- 172.21.15.103:0/1220760528 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c03f070 con 0x7f2844074eb0 2024-09-16T02:03:04.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.269+0000 7f2840c00640 1 -- 172.21.15.103:0/1220760528 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f282c031ca0 con 0x7f2844074eb0 2024-09-16T02:03:04.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.269+0000 7f284a600640 1 -- 172.21.15.103:0/1220760528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284419f2c0 con 0x7f2844074eb0 2024-09-16T02:03:04.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.269+0000 7f2840c00640 1 -- 172.21.15.103:0/1220760528 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c035030 con 0x7f2844074eb0 2024-09-16T02:03:04.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.269+0000 7f284a600640 1 -- 172.21.15.103:0/1220760528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28441ab610 con 0x7f2844074eb0 2024-09-16T02:03:04.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.270+0000 7f284a600640 1 -- 172.21.15.103:0/1220760528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2844074970 con 0x7f2844074eb0 2024-09-16T02:03:04.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.272+0000 7f2840c00640 1 -- 172.21.15.103:0/1220760528 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f282c042070 con 0x7f2844074eb0 2024-09-16T02:03:04.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.273+0000 7f2840c00640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f28280a3240 0x7f28280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:04.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.273+0000 7f2840c00640 1 -- 172.21.15.103:0/1220760528 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f282c0f0050 con 0x7f2844074eb0 2024-09-16T02:03:04.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.274+0000 7f2843400640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f28280a3240 0x7f28280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:04.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.274+0000 7f2843400640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f28280a3240 0x7f28280a5700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f284419fa40 tx=0x7f2834005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:04.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.275+0000 7f2840c00640 1 -- 172.21.15.103:0/1220760528 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f282c049030 con 0x7f2844074eb0 2024-09-16T02:03:04.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.487+0000 7f284a600640 1 -- 172.21.15.103:0/1220760528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2844105000 con 0x7f2844074eb0 2024-09-16T02:03:04.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.487+0000 7f2840c00640 1 -- 172.21.15.103:0/1220760528 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7f282c0bd230 con 0x7f2844074eb0 2024-09-16T02:03:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:04.489 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":5,"btime":"2024-09-16T02:03:03:492476+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-09-16T02:03:04.489 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 5 2024-09-16T02:03:04.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 -- 172.21.15.103:0/1220760528 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f28280a3240 msgr2=0x7f28280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:04.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f28280a3240 0x7f28280a5700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f284419fa40 tx=0x7f2834005f90 comp rx=0 tx=0).stop 2024-09-16T02:03:04.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 -- 172.21.15.103:0/1220760528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844074eb0 msgr2=0x7f28441a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:04.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844074eb0 0x7f28441a4c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f282c004770 tx=0x7f282c003c10 comp rx=0 tx=0).stop 2024-09-16T02:03:04.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 -- 172.21.15.103:0/1220760528 shutdown_connections 2024-09-16T02:03:04.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f28280a3240 0x7f28280a5700 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:04.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2844112b90 0x7f284419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:04.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844074eb0 0x7f28441a4c80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:04.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 --2- 172.21.15.103:0/1220760528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2844074590 0x7f28441a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:04.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 -- 172.21.15.103:0/1220760528 >> 172.21.15.103:0/1220760528 conn(0x7f28440fd8c0 msgr2=0x7f28440ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:04.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 -- 172.21.15.103:0/1220760528 shutdown_connections 2024-09-16T02:03:04.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:04.490+0000 7f284a600640 1 -- 172.21.15.103:0/1220760528 wait complete. 2024-09-16T02:03:04.750 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.103:6836/2740838933,v1:172.21.15.103:6837/2740838933] up:boot 2024-09-16T02:03:04.750 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: fsmap 4 up:standby 2024-09-16T02:03:04.750 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T02:03:04.750 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:04.750 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:04.750 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:04.750 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:04.751 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:04.751 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:04.751 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: Deploying daemon mds.e on smithi136 2024-09-16T02:03:04.751 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:04 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1220760528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:04.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.103:6836/2740838933,v1:172.21.15.103:6837/2740838933] up:boot 2024-09-16T02:03:04.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: fsmap 4 up:standby 2024-09-16T02:03:04.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T02:03:04.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:04.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:04.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:04.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:04.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:04.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:04.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: Deploying daemon mds.e on smithi136 2024-09-16T02:03:04.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:04 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1220760528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:04.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.103:6836/2740838933,v1:172.21.15.103:6837/2740838933] up:boot 2024-09-16T02:03:04.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: fsmap 4 up:standby 2024-09-16T02:03:04.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T02:03:04.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:04.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:04.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:04.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:04.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:04.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:04.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: Deploying daemon mds.e on smithi136 2024-09-16T02:03:04.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:04 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1220760528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:05.523 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:03:05 smithi136 systemd[1]: Starting Ceph mds.e for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:03:05.873 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:05 smithi136 ceph-mon[35323]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:05.874 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:03:05 smithi136 podman[65442]: 2024-09-16 02:03:05.636946008 +0000 UTC m=+0.101592775 container create f79535200b0ae785a43503e5ffdb094c38bcfdd7ffe8ecf87c07e2fe722f4f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T02:03:05.874 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:03:05 smithi136 podman[65442]: 2024-09-16 02:03:05.551325421 +0000 UTC m=+0.015972191 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:03:05.874 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:03:05 smithi136 podman[65442]: 2024-09-16 02:03:05.872925632 +0000 UTC m=+0.337572391 container init f79535200b0ae785a43503e5ffdb094c38bcfdd7ffe8ecf87c07e2fe722f4f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:03:06.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:05 smithi103 ceph-mon[30398]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:06.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:05 smithi172 ceph-mon[35340]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:06.273 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:03:05 smithi136 podman[65442]: 2024-09-16 02:03:05.876052935 +0000 UTC m=+0.340699704 container start f79535200b0ae785a43503e5ffdb094c38bcfdd7ffe8ecf87c07e2fe722f4f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T02:03:06.273 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:03:05 smithi136 ceph-mds[65461]: -- 172.21.15.136:0/2497543583 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe3dcd9860 con 0x55fe3dcb6800 2024-09-16T02:03:06.273 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:03:05 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e[65457]: starting mds.e at 2024-09-16T02:03:06.273 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:03:05 smithi136 bash[65442]: f79535200b0ae785a43503e5ffdb094c38bcfdd7ffe8ecf87c07e2fe722f4f0b 2024-09-16T02:03:06.273 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:03:05 smithi136 systemd[1]: Started Ceph mds.e for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:03:07.089 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:03:07.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:07.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:07.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:07.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:07.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:07.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:07.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: Deploying daemon mds.f on smithi172 2024-09-16T02:03:07.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:07.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.136:6834/2566717089,v1:172.21.15.136:6835/2566717089] up:boot 2024-09-16T02:03:07.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: fsmap 5 up:standby 2024-09-16T02:03:07.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:07 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:03:07.470 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:07.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:07.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:07.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:07.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:07.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:07.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:07.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: Deploying daemon mds.f on smithi172 2024-09-16T02:03:07.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:07.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.136:6834/2566717089,v1:172.21.15.136:6835/2566717089] up:boot 2024-09-16T02:03:07.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: fsmap 5 up:standby 2024-09-16T02:03:07.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:07 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:03:07.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:07.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:07.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:07.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:07.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:07.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:07.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: Deploying daemon mds.f on smithi172 2024-09-16T02:03:07.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:07.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.136:6834/2566717089,v1:172.21.15.136:6835/2566717089] up:boot 2024-09-16T02:03:07.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: fsmap 5 up:standby 2024-09-16T02:03:07.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:07 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:03:08.045 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:03:07 smithi172 systemd[1]: Starting Ceph mds.f for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:03:08.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.090+0000 7f24aaa00640 1 -- 172.21.15.103:0/434633307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24a4108f60 msgr2=0x7f24a4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:08.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.090+0000 7f24aaa00640 1 --2- 172.21.15.103:0/434633307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24a4108f60 0x7f24a4110ae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f249c009be0 tx=0x7f249c02a6c0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.090+0000 7f24aaa00640 1 -- 172.21.15.103:0/434633307 shutdown_connections 2024-09-16T02:03:08.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.090+0000 7f24aaa00640 1 --2- 172.21.15.103:0/434633307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24a4108f60 0x7f24a4110ae0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.090+0000 7f24aaa00640 1 --2- 172.21.15.103:0/434633307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24a41047e0 0x7f24a4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.090+0000 7f24aaa00640 1 --2- 172.21.15.103:0/434633307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4103e30 0x7f24a4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.090+0000 7f24aaa00640 1 -- 172.21.15.103:0/434633307 >> 172.21.15.103:0/434633307 conn(0x7f24a40ff910 msgr2=0x7f24a4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:08.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.090+0000 7f24aaa00640 1 -- 172.21.15.103:0/434633307 shutdown_connections 2024-09-16T02:03:08.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.090+0000 7f24aaa00640 1 -- 172.21.15.103:0/434633307 wait complete. 2024-09-16T02:03:08.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.091+0000 7f24aaa00640 1 Processor -- start 2024-09-16T02:03:08.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.091+0000 7f24aaa00640 1 -- start start 2024-09-16T02:03:08.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.091+0000 7f24aaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24a4103e30 0x7f24a419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:08.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24a41047e0 0x7f24a419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:08.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aaa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4108f60 0x7f24a419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:08.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aaa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4113f70 con 0x7f24a41047e0 2024-09-16T02:03:08.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aaa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4113df0 con 0x7f24a4103e30 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aaa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41140f0 con 0x7f24a4108f60 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24a8c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24a41047e0 0x7f24a419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aa000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4108f60 0x7f24a419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aa000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4108f60 0x7f24a419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:34256/0 (socket says 172.21.15.103:34256) 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aa000640 1 -- 172.21.15.103:0/1134589851 learned_addr learned my addr 172.21.15.103:0/1134589851 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24a9600640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24a4103e30 0x7f24a419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aa000640 1 -- 172.21.15.103:0/1134589851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24a4103e30 msgr2=0x7f24a419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aa000640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24a4103e30 0x7f24a419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aa000640 1 -- 172.21.15.103:0/1134589851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24a41047e0 msgr2=0x7f24a419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aa000640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24a41047e0 0x7f24a419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.092+0000 7f24aa000640 1 -- 172.21.15.103:0/1134589851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a419f420 con 0x7f24a4108f60 2024-09-16T02:03:08.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.093+0000 7f24aa000640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4108f60 0x7f24a419ed40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f249c034040 tx=0x7f249c0305b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:08.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.093+0000 7f24a2000640 1 -- 172.21.15.103:0/1134589851 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c041020 con 0x7f24a4108f60 2024-09-16T02:03:08.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.093+0000 7f24aaa00640 1 -- 172.21.15.103:0/1134589851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a419f6b0 con 0x7f24a4108f60 2024-09-16T02:03:08.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.093+0000 7f24a2000640 1 -- 172.21.15.103:0/1134589851 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f249c037040 con 0x7f24a4108f60 2024-09-16T02:03:08.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.093+0000 7f24a2000640 1 -- 172.21.15.103:0/1134589851 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c036070 con 0x7f24a4108f60 2024-09-16T02:03:08.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.093+0000 7f24aaa00640 1 -- 172.21.15.103:0/1134589851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a41a6ff0 con 0x7f24a4108f60 2024-09-16T02:03:08.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.094+0000 7f24aaa00640 1 -- 172.21.15.103:0/1134589851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a4105310 con 0x7f24a4108f60 2024-09-16T02:03:08.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.096+0000 7f24a2000640 1 -- 172.21.15.103:0/1134589851 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f249c044020 con 0x7f24a4108f60 2024-09-16T02:03:08.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.097+0000 7f24a2000640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f248c0a3240 0x7f248c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:08.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.097+0000 7f24a2000640 1 -- 172.21.15.103:0/1134589851 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f249c0eff00 con 0x7f24a4108f60 2024-09-16T02:03:08.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.097+0000 7f24a9600640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f248c0a3240 0x7f248c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:08.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.098+0000 7f24a9600640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f248c0a3240 0x7f248c0a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2498005fd0 tx=0x7f2498009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:08.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.101+0000 7f24a2000640 1 -- 172.21.15.103:0/1134589851 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f249c04b020 con 0x7f24a4108f60 2024-09-16T02:03:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.311+0000 7f24aaa00640 1 -- 172.21.15.103:0/1134589851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24a4104210 con 0x7f24a4108f60 2024-09-16T02:03:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.312+0000 7f24a2000640 1 -- 172.21.15.103:0/1134589851 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4750 (secure 0 0 0) 0x7f24a4104210 con 0x7f24a4108f60 2024-09-16T02:03:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:08.313 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":6,"btime":"2024-09-16T02:03:06:621326+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-16T02:03:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 6 2024-09-16T02:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 -- 172.21.15.103:0/1134589851 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f248c0a3240 msgr2=0x7f248c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f248c0a3240 0x7f248c0a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2498005fd0 tx=0x7f2498009260 comp rx=0 tx=0).stop 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 -- 172.21.15.103:0/1134589851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4108f60 msgr2=0x7f24a419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4108f60 0x7f24a419ed40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f249c034040 tx=0x7f249c0305b0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 -- 172.21.15.103:0/1134589851 shutdown_connections 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f248c0a3240 0x7f248c0a5700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4108f60 0x7f24a419ed40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24a41047e0 0x7f24a419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 --2- 172.21.15.103:0/1134589851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24a4103e30 0x7f24a419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 -- 172.21.15.103:0/1134589851 >> 172.21.15.103:0/1134589851 conn(0x7f24a40ff910 msgr2=0x7f24a4100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.314+0000 7f24aaa00640 1 -- 172.21.15.103:0/1134589851 shutdown_connections 2024-09-16T02:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:08.315+0000 7f24aaa00640 1 -- 172.21.15.103:0/1134589851 wait complete. 2024-09-16T02:03:08.334 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:03:08 smithi172 podman[65520]: 2024-09-16 02:03:07.982908896 +0000 UTC m=+0.015925425 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:03:08.334 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:03:08 smithi172 podman[65520]: 2024-09-16 02:03:08.085507455 +0000 UTC m=+0.118523974 container create 54d1f2756558e70ca41c922fe8a9a1ce238c88f4e23b1aa72675c5c6fec6944c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-16T02:03:08.334 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:03:08 smithi172 podman[65520]: 2024-09-16 02:03:08.329943542 +0000 UTC m=+0.362960065 container init 54d1f2756558e70ca41c922fe8a9a1ce238c88f4e23b1aa72675c5c6fec6944c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T02:03:08.334 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:03:08 smithi172 podman[65520]: 2024-09-16 02:03:08.33331768 +0000 UTC m=+0.366334200 container start 54d1f2756558e70ca41c922fe8a9a1ce238c88f4e23b1aa72675c5c6fec6944c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T02:03:08.655 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:08 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1134589851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:08.693 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:08 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1134589851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:08.696 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:08 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1134589851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:08.696 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:03:08 smithi172 ceph-mds[65541]: -- 172.21.15.172:0/2729287058 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb79cad860 con 0x55bb79c8b400 2024-09-16T02:03:08.696 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:03:08 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-16T02:03:08.696 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:03:08 smithi172 bash[65520]: 54d1f2756558e70ca41c922fe8a9a1ce238c88f4e23b1aa72675c5c6fec6944c 2024-09-16T02:03:08.697 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:03:08 smithi172 systemd[1]: Started Ceph mds.f for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:03:09.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.172:6834/2720370264,v1:172.21.15.172:6835/2720370264] up:boot 2024-09-16T02:03:09.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: fsmap 6 up:standby 2024-09-16T02:03:09.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:03:09.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:09.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:09.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:09.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:09.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:09.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:09.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:09.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:09 smithi103 ceph-mon[30398]: Deploying daemon mds.c on smithi172 2024-09-16T02:03:09.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.172:6834/2720370264,v1:172.21.15.172:6835/2720370264] up:boot 2024-09-16T02:03:09.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: fsmap 6 up:standby 2024-09-16T02:03:09.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:03:09.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:09.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:09.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:09.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:09.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:09.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:09.775 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:09.775 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:09 smithi136 ceph-mon[35323]: Deploying daemon mds.c on smithi172 2024-09-16T02:03:09.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.172:6834/2720370264,v1:172.21.15.172:6835/2720370264] up:boot 2024-09-16T02:03:09.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: fsmap 6 up:standby 2024-09-16T02:03:09.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:03:09.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:09.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:09.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:09.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:09.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:09.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:09.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:09.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:09 smithi172 ceph-mon[35340]: Deploying daemon mds.c on smithi172 2024-09-16T02:03:10.548 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:03:10 smithi172 systemd[1]: Starting Ceph mds.c for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:03:10.827 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:03:10 smithi172 podman[65837]: 2024-09-16 02:03:10.572303088 +0000 UTC m=+0.097190014 container create 1a8f64a9b35339a0060835a6d7abc9071369d59b0deeccec716597ec678a001b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T02:03:10.828 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:03:10 smithi172 podman[65837]: 2024-09-16 02:03:10.490573824 +0000 UTC m=+0.015460751 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:03:10.828 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:03:10 smithi172 podman[65837]: 2024-09-16 02:03:10.800038271 +0000 UTC m=+0.324925199 container init 1a8f64a9b35339a0060835a6d7abc9071369d59b0deeccec716597ec678a001b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-16T02:03:10.828 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:03:10 smithi172 podman[65837]: 2024-09-16 02:03:10.802906476 +0000 UTC m=+0.327793406 container start 1a8f64a9b35339a0060835a6d7abc9071369d59b0deeccec716597ec678a001b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-16T02:03:11.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:03:11.295 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:03:10 smithi172 ceph-mds[65856]: -- 172.21.15.172:0/3912587927 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604fa287860 con 0x5604fa264800 2024-09-16T02:03:11.295 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:03:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c[65852]: starting mds.c at 2024-09-16T02:03:11.295 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:03:10 smithi172 bash[65837]: 1a8f64a9b35339a0060835a6d7abc9071369d59b0deeccec716597ec678a001b 2024-09-16T02:03:11.296 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:03:10 smithi172 systemd[1]: Started Ceph mds.c for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:03:11.441 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:11.693 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:11 smithi136 ceph-mon[35323]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:11.693 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:11.693 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:11.693 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:11.693 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:11.693 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:11.694 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:11 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:11.752 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:11 smithi103 ceph-mon[30398]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:11.752 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:11.752 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:11.752 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:11.753 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:11.753 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:11.753 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:11 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:11 smithi172 ceph-mon[35340]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:11.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:11 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:12.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.009+0000 7f5abec00640 1 -- 172.21.15.103:0/2016519377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8077040 msgr2=0x7f5ab81124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:12.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.009+0000 7f5abec00640 1 --2- 172.21.15.103:0/2016519377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8077040 0x7f5ab81124f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009900 tx=0x7f5aa402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.009+0000 7f5abec00640 1 -- 172.21.15.103:0/2016519377 shutdown_connections 2024-09-16T02:03:12.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.009+0000 7f5abec00640 1 --2- 172.21.15.103:0/2016519377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab8112a30 0x7f5ab8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.009+0000 7f5abec00640 1 --2- 172.21.15.103:0/2016519377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8077040 0x7f5ab81124f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.009+0000 7f5abec00640 1 --2- 172.21.15.103:0/2016519377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab8076720 0x7f5ab8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.009+0000 7f5abec00640 1 -- 172.21.15.103:0/2016519377 >> 172.21.15.103:0/2016519377 conn(0x7f5ab80ffa90 msgr2=0x7f5ab8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:12.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.009+0000 7f5abec00640 1 -- 172.21.15.103:0/2016519377 shutdown_connections 2024-09-16T02:03:12.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.009+0000 7f5abec00640 1 -- 172.21.15.103:0/2016519377 wait complete. 2024-09-16T02:03:12.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.010+0000 7f5abec00640 1 Processor -- start 2024-09-16T02:03:12.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.010+0000 7f5abec00640 1 -- start start 2024-09-16T02:03:12.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.011+0000 7f5abec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab8076720 0x7f5ab81a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:12.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.011+0000 7f5abec00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab8077040 0x7f5ab81a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:12.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.011+0000 7f5abec00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8112a30 0x7f5ab819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:12.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.011+0000 7f5abec00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8118600 con 0x7f5ab8112a30 2024-09-16T02:03:12.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.011+0000 7f5abec00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8118480 con 0x7f5ab8077040 2024-09-16T02:03:12.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.011+0000 7f5abec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8118780 con 0x7f5ab8076720 2024-09-16T02:03:12.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.011+0000 7f5ab7e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab8076720 0x7f5ab81a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:12.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.011+0000 7f5ab7e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab8076720 0x7f5ab81a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:34276/0 (socket says 172.21.15.103:34276) 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.011+0000 7f5abcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8112a30 0x7f5ab819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.011+0000 7f5ab7e00640 1 -- 172.21.15.103:0/79415542 learned_addr learned my addr 172.21.15.103:0/79415542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5ab7400640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab8077040 0x7f5ab81a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5abcc00640 1 -- 172.21.15.103:0/79415542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab8076720 msgr2=0x7f5ab81a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5abcc00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab8076720 0x7f5ab81a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5abcc00640 1 -- 172.21.15.103:0/79415542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab8077040 msgr2=0x7f5ab81a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5abcc00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab8077040 0x7f5ab81a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5abcc00640 1 -- 172.21.15.103:0/79415542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab819eff0 con 0x7f5ab8112a30 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5ab7e00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab8076720 0x7f5ab81a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5ab7400640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab8077040 0x7f5ab81a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5abcc00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8112a30 0x7f5ab819e8c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ab807aa10 tx=0x7f5ab000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5ab4c00640 1 -- 172.21.15.103:0/79415542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0017070 con 0x7f5ab8112a30 2024-09-16T02:03:12.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5ab4c00640 1 -- 172.21.15.103:0/79415542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab00047a0 con 0x7f5ab8112a30 2024-09-16T02:03:12.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5abec00640 1 -- 172.21.15.103:0/79415542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab819f2e0 con 0x7f5ab8112a30 2024-09-16T02:03:12.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5ab4c00640 1 -- 172.21.15.103:0/79415542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0009ce0 con 0x7f5ab8112a30 2024-09-16T02:03:12.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.012+0000 7f5abec00640 1 -- 172.21.15.103:0/79415542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab81ab5d0 con 0x7f5ab8112a30 2024-09-16T02:03:12.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.013+0000 7f5abec00640 1 -- 172.21.15.103:0/79415542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab8076b00 con 0x7f5ab8112a30 2024-09-16T02:03:12.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.017+0000 7f5ab4c00640 1 -- 172.21.15.103:0/79415542 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5ab0004950 con 0x7f5ab8112a30 2024-09-16T02:03:12.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.018+0000 7f5ab4c00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5a9c0a3240 0x7f5a9c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:12.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.018+0000 7f5ab4c00640 1 -- 172.21.15.103:0/79415542 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f5ab00cf140 con 0x7f5ab8112a30 2024-09-16T02:03:12.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.018+0000 7f5ab7e00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5a9c0a3240 0x7f5a9c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:12.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.019+0000 7f5ab7e00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5a9c0a3240 0x7f5a9c0a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ab819fa00 tx=0x7f5aa8009720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:12.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.020+0000 7f5ab4c00640 1 -- 172.21.15.103:0/79415542 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ab009c260 con 0x7f5ab8112a30 2024-09-16T02:03:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.229+0000 7f5abec00640 1 -- 172.21.15.103:0/79415542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ab802d7c0 con 0x7f5ab8112a30 2024-09-16T02:03:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.230+0000 7f5ab4c00640 1 -- 172.21.15.103:0/79415542 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6406 (secure 0 0 0) 0x7f5ab009bb60 con 0x7f5ab8112a30 2024-09-16T02:03:12.232 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:12.232 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":8,"btime":"2024-09-16T02:03:11:494115+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-16T02:03:12.234 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 8 2024-09-16T02:03:12.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.232+0000 7f5abec00640 1 -- 172.21.15.103:0/79415542 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5a9c0a3240 msgr2=0x7f5a9c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:12.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.232+0000 7f5abec00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5a9c0a3240 0x7f5a9c0a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ab819fa00 tx=0x7f5aa8009720 comp rx=0 tx=0).stop 2024-09-16T02:03:12.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.232+0000 7f5abec00640 1 -- 172.21.15.103:0/79415542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8112a30 msgr2=0x7f5ab819e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:12.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.232+0000 7f5abec00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8112a30 0x7f5ab819e8c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ab807aa10 tx=0x7f5ab000dae0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.232+0000 7f5abec00640 1 -- 172.21.15.103:0/79415542 shutdown_connections 2024-09-16T02:03:12.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.232+0000 7f5abec00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f5a9c0a3240 0x7f5a9c0a5700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.232+0000 7f5abec00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8112a30 0x7f5ab819e8c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.232+0000 7f5abec00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab8077040 0x7f5ab81a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.232+0000 7f5abec00640 1 --2- 172.21.15.103:0/79415542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab8076720 0x7f5ab81a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:12.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.232+0000 7f5abec00640 1 -- 172.21.15.103:0/79415542 >> 172.21.15.103:0/79415542 conn(0x7f5ab80ffa90 msgr2=0x7f5ab8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:12.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.233+0000 7f5abec00640 1 -- 172.21.15.103:0/79415542 shutdown_connections 2024-09-16T02:03:12.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:12.233+0000 7f5abec00640 1 -- 172.21.15.103:0/79415542 wait complete. 2024-09-16T02:03:12.657 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:12 smithi136 ceph-mon[35323]: Deploying daemon mds.h on smithi136 2024-09-16T02:03:12.657 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:12 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.172:6836/1584538248,v1:172.21.15.172:6837/1584538248] up:boot 2024-09-16T02:03:12.657 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:12 smithi136 ceph-mon[35323]: fsmap 7 up:standby 2024-09-16T02:03:12.657 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:12 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:03:12.657 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:12 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/79415542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:12.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:12 smithi172 ceph-mon[35340]: Deploying daemon mds.h on smithi136 2024-09-16T02:03:12.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:12 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.172:6836/1584538248,v1:172.21.15.172:6837/1584538248] up:boot 2024-09-16T02:03:12.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:12 smithi172 ceph-mon[35340]: fsmap 7 up:standby 2024-09-16T02:03:12.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:12 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:03:12.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:12 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/79415542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:12.910 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:12 smithi103 ceph-mon[30398]: Deploying daemon mds.h on smithi136 2024-09-16T02:03:12.911 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:12 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.172:6836/1584538248,v1:172.21.15.172:6837/1584538248] up:boot 2024-09-16T02:03:12.911 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:12 smithi103 ceph-mon[30398]: fsmap 7 up:standby 2024-09-16T02:03:12.911 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:12 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:03:12.911 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:12 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/79415542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:13.023 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:03:12 smithi136 systemd[1]: Starting Ceph mds.h for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:03:13.321 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:03:13 smithi136 podman[65749]: 2024-09-16 02:03:13.057193328 +0000 UTC m=+0.112101836 container create 5bd2a4f82897dc0acae37cdafe5b797b13ec825f73b14a9ac9410eab76c695ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T02:03:13.321 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:03:13 smithi136 podman[65749]: 2024-09-16 02:03:12.960931731 +0000 UTC m=+0.015840241 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:03:13.321 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:03:13 smithi136 podman[65749]: 2024-09-16 02:03:13.293396545 +0000 UTC m=+0.348305055 container init 5bd2a4f82897dc0acae37cdafe5b797b13ec825f73b14a9ac9410eab76c695ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T02:03:13.321 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:03:13 smithi136 podman[65749]: 2024-09-16 02:03:13.296366834 +0000 UTC m=+0.351275345 container start 5bd2a4f82897dc0acae37cdafe5b797b13ec825f73b14a9ac9410eab76c695ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T02:03:13.773 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:03:13 smithi136 ceph-mds[65769]: -- 172.21.15.136:0/1523890531 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558625275860 con 0x558625253400 2024-09-16T02:03:13.773 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:03:13 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h[65765]: starting mds.h at 2024-09-16T02:03:13.773 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:03:13 smithi136 bash[65749]: 5bd2a4f82897dc0acae37cdafe5b797b13ec825f73b14a9ac9410eab76c695ff 2024-09-16T02:03:13.773 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:03:13 smithi136 systemd[1]: Started Ceph mds.h for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:03:13.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:13 smithi136 ceph-mon[35323]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:13.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:13.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:13.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:13.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:13.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:13.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:13 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:13 smithi172 ceph-mon[35340]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:13 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:13 smithi103 ceph-mon[30398]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:13 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:14.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:14 smithi172 ceph-mon[35340]: Deploying daemon mds.i on smithi172 2024-09-16T02:03:14.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:14 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] up:boot 2024-09-16T02:03:14.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:14 smithi172 ceph-mon[35340]: fsmap 8 up:standby 2024-09-16T02:03:14.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:14 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:03:14.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:03:15.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:14 smithi103 ceph-mon[30398]: Deploying daemon mds.i on smithi172 2024-09-16T02:03:15.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:14 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] up:boot 2024-09-16T02:03:15.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:14 smithi103 ceph-mon[30398]: fsmap 8 up:standby 2024-09-16T02:03:15.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:14 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:03:15.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:14 smithi136 ceph-mon[35323]: Deploying daemon mds.i on smithi172 2024-09-16T02:03:15.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:14 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] up:boot 2024-09-16T02:03:15.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:14 smithi136 ceph-mon[35323]: fsmap 8 up:standby 2024-09-16T02:03:15.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:14 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:03:15.254 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:03:15 smithi172 systemd[1]: Starting Ceph mds.i for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:03:15.329 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:15.529 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:03:15 smithi172 podman[66134]: 2024-09-16 02:03:15.367534816 +0000 UTC m=+0.015862518 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:03:15.529 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:03:15 smithi172 podman[66134]: 2024-09-16 02:03:15.467488837 +0000 UTC m=+0.115816542 container create bb31bc2221b4b62047833ffc76dd151aa14eafe87630a44607e7db3743d0cf00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T02:03:15.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:15 smithi103 ceph-mon[30398]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:15.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:15 smithi172 ceph-mon[35340]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:15.795 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:03:15 smithi172 podman[66134]: 2024-09-16 02:03:15.678564513 +0000 UTC m=+0.326892225 container init bb31bc2221b4b62047833ffc76dd151aa14eafe87630a44607e7db3743d0cf00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T02:03:15.795 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:03:15 smithi172 podman[66134]: 2024-09-16 02:03:15.681398004 +0000 UTC m=+0.329725707 container start bb31bc2221b4b62047833ffc76dd151aa14eafe87630a44607e7db3743d0cf00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:03:15.795 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:03:15 smithi172 ceph-mds[66153]: -- 172.21.15.172:0/1825606188 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559789ca5860 con 0x559789c83000 2024-09-16T02:03:15.795 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:03:15 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i[66149]: starting mds.i at 2024-09-16T02:03:15.795 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:03:15 smithi172 bash[66134]: bb31bc2221b4b62047833ffc76dd151aa14eafe87630a44607e7db3743d0cf00 2024-09-16T02:03:15.795 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:03:15 smithi172 systemd[1]: Started Ceph mds.i for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:03:15.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.930+0000 7ff049e00640 1 -- 172.21.15.103:0/3554768252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff044074e30 msgr2=0x7ff04410df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:15.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.930+0000 7ff049e00640 1 --2- 172.21.15.103:0/3554768252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff044074e30 0x7ff04410df00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009860 tx=0x7ff02c02a3d0 comp rx=0 tx=0).stop 2024-09-16T02:03:15.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.930+0000 7ff049e00640 1 -- 172.21.15.103:0/3554768252 shutdown_connections 2024-09-16T02:03:15.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.930+0000 7ff049e00640 1 --2- 172.21.15.103:0/3554768252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04410e440 0x7ff044110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:15.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.930+0000 7ff049e00640 1 --2- 172.21.15.103:0/3554768252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff044074e30 0x7ff04410df00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:15.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.930+0000 7ff049e00640 1 --2- 172.21.15.103:0/3554768252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff044074510 0x7ff0440748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:15.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.930+0000 7ff049e00640 1 -- 172.21.15.103:0/3554768252 >> 172.21.15.103:0/3554768252 conn(0x7ff0440fd8c0 msgr2=0x7ff0440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:15.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.930+0000 7ff049e00640 1 -- 172.21.15.103:0/3554768252 shutdown_connections 2024-09-16T02:03:15.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.930+0000 7ff049e00640 1 -- 172.21.15.103:0/3554768252 wait complete. 2024-09-16T02:03:15.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.931+0000 7ff049e00640 1 Processor -- start 2024-09-16T02:03:15.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.931+0000 7ff049e00640 1 -- start start 2024-09-16T02:03:15.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.931+0000 7ff049e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff044074510 0x7ff04419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:15.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.931+0000 7ff049e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff044074e30 0x7ff04419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:15.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.931+0000 7ff049e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04410e440 0x7ff04419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:15.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.931+0000 7ff049e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff044114160 con 0x7ff044074e30 2024-09-16T02:03:15.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.931+0000 7ff049e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff044113fe0 con 0x7ff044074510 2024-09-16T02:03:15.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.931+0000 7ff049e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441142e0 con 0x7ff04410e440 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.932+0000 7ff03a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff044074e30 0x7ff04419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.932+0000 7ff03a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff044074e30 0x7ff04419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40896/0 (socket says 172.21.15.103:40896) 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.932+0000 7ff03a000640 1 -- 172.21.15.103:0/2344832707 learned_addr learned my addr 172.21.15.103:0/2344832707 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.932+0000 7ff043400640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04410e440 0x7ff04419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.932+0000 7ff042a00640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff044074510 0x7ff04419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.932+0000 7ff043400640 1 -- 172.21.15.103:0/2344832707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff044074510 msgr2=0x7ff04419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.932+0000 7ff043400640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff044074510 0x7ff04419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.932+0000 7ff043400640 1 -- 172.21.15.103:0/2344832707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff044074e30 msgr2=0x7ff04419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.932+0000 7ff043400640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff044074e30 0x7ff04419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.932+0000 7ff043400640 1 -- 172.21.15.103:0/2344832707 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04419f670 con 0x7ff04410e440 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.933+0000 7ff043400640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04410e440 0x7ff04419ef90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff03c005fd0 tx=0x7ff03c00eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:15.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.933+0000 7ff048a00640 1 -- 172.21.15.103:0/2344832707 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c004220 con 0x7ff04410e440 2024-09-16T02:03:15.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.933+0000 7ff048a00640 1 -- 172.21.15.103:0/2344832707 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff03c0043c0 con 0x7ff04410e440 2024-09-16T02:03:15.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.933+0000 7ff049e00640 1 -- 172.21.15.103:0/2344832707 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04419f960 con 0x7ff04410e440 2024-09-16T02:03:15.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.933+0000 7ff048a00640 1 -- 172.21.15.103:0/2344832707 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c004a00 con 0x7ff04410e440 2024-09-16T02:03:15.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.933+0000 7ff049e00640 1 -- 172.21.15.103:0/2344832707 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0441a7240 con 0x7ff04410e440 2024-09-16T02:03:15.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.934+0000 7ff049e00640 1 -- 172.21.15.103:0/2344832707 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff020005290 con 0x7ff04410e440 2024-09-16T02:03:15.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.936+0000 7ff048a00640 1 -- 172.21.15.103:0/2344832707 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff03c004ba0 con 0x7ff04410e440 2024-09-16T02:03:15.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.937+0000 7ff048a00640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0280a32c0 0x7ff0280a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:15.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.937+0000 7ff048a00640 1 -- 172.21.15.103:0/2344832707 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7ff03c0cf150 con 0x7ff04410e440 2024-09-16T02:03:15.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.937+0000 7ff042a00640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0280a32c0 0x7ff0280a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:15.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.938+0000 7ff042a00640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0280a32c0 0x7ff0280a5780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff034005fd0 tx=0x7ff034009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:15.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:15.941+0000 7ff048a00640 1 -- 172.21.15.103:0/2344832707 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff03c09c270 con 0x7ff04410e440 2024-09-16T02:03:16.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:15 smithi136 ceph-mon[35323]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.160+0000 7ff049e00640 1 -- 172.21.15.103:0/2344832707 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff020005580 con 0x7ff04410e440 2024-09-16T02:03:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.160+0000 7ff048a00640 1 -- 172.21.15.103:0/2344832707 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7231 (secure 0 0 0) 0x7ff03c09bb70 con 0x7ff04410e440 2024-09-16T02:03:16.162 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:16.162 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":9,"btime":"2024-09-16T02:03:13:507799+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T02:03:16.162 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 9 2024-09-16T02:03:16.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 -- 172.21.15.103:0/2344832707 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0280a32c0 msgr2=0x7ff0280a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:16.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0280a32c0 0x7ff0280a5780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff034005fd0 tx=0x7ff034009870 comp rx=0 tx=0).stop 2024-09-16T02:03:16.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 -- 172.21.15.103:0/2344832707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04410e440 msgr2=0x7ff04419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:16.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04410e440 0x7ff04419ef90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff03c005fd0 tx=0x7ff03c00eba0 comp rx=0 tx=0).stop 2024-09-16T02:03:16.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 -- 172.21.15.103:0/2344832707 shutdown_connections 2024-09-16T02:03:16.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0280a32c0 0x7ff0280a5780 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:16.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04410e440 0x7ff04419ef90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:16.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff044074e30 0x7ff04419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:16.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 --2- 172.21.15.103:0/2344832707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff044074510 0x7ff04419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:16.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 -- 172.21.15.103:0/2344832707 >> 172.21.15.103:0/2344832707 conn(0x7ff0440fd8c0 msgr2=0x7ff0440fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:16.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 -- 172.21.15.103:0/2344832707 shutdown_connections 2024-09-16T02:03:16.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:16.163+0000 7ff049e00640 1 -- 172.21.15.103:0/2344832707 wait complete. 2024-09-16T02:03:17.142 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:17.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:17.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:17.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:17.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:17.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:17.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: Deploying daemon mds.g on smithi103 2024-09-16T02:03:17.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2344832707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:17.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:17.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] up:boot 2024-09-16T02:03:17.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: fsmap 9 up:standby 2024-09-16T02:03:17.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:16 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:03:17.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: Deploying daemon mds.g on smithi103 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2344832707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] up:boot 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: fsmap 9 up:standby 2024-09-16T02:03:17.274 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:16 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:03:17.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:17.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:17.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:17.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:17.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:17.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:17.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: Deploying daemon mds.g on smithi103 2024-09-16T02:03:17.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2344832707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:17.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:17.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] up:boot 2024-09-16T02:03:17.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: fsmap 9 up:standby 2024-09-16T02:03:17.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:16 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:03:18.957 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:03:19.374 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:03:19 smithi103 podman[71270]: 2024-09-16 02:03:19.14412995 +0000 UTC m=+0.166827702 container create e4778ffffc52ec1cfe11072b1d98a23ab8b58809e657797b3f88b3b60005882a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-16T02:03:19.374 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:03:19 smithi103 podman[71270]: 2024-09-16 02:03:19.372355966 +0000 UTC m=+0.395053719 container init e4778ffffc52ec1cfe11072b1d98a23ab8b58809e657797b3f88b3b60005882a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-g, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T02:03:19.586 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:19.726 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:19 smithi103 ceph-mon[30398]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:19.726 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:03:19 smithi103 podman[71270]: 2024-09-16 02:03:19.378979277 +0000 UTC m=+0.401677030 container start e4778ffffc52ec1cfe11072b1d98a23ab8b58809e657797b3f88b3b60005882a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T02:03:19.726 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:03:19 smithi103 ceph-mds[71332]: -- 172.21.15.103:0/1990838553 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2b7519860 con 0x55c2b74f7000 2024-09-16T02:03:19.726 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:03:19 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-g[71321]: starting mds.g at 2024-09-16T02:03:19.727 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:03:19 smithi103 bash[71270]: e4778ffffc52ec1cfe11072b1d98a23ab8b58809e657797b3f88b3b60005882a 2024-09-16T02:03:19.727 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:03:19 smithi103 systemd[1]: Started Ceph mds.g for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:03:19.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:19 smithi136 ceph-mon[35323]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:19.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:19 smithi172 ceph-mon[35340]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.150+0000 7f274b600640 1 -- 172.21.15.103:0/2520305476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2744106f00 msgr2=0x7f274410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.150+0000 7f274b600640 1 --2- 172.21.15.103:0/2520305476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2744106f00 0x7f274410ea80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f273c009ca0 tx=0x7f273c02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.150+0000 7f274b600640 1 -- 172.21.15.103:0/2520305476 shutdown_connections 2024-09-16T02:03:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.150+0000 7f274b600640 1 --2- 172.21.15.103:0/2520305476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2744106f00 0x7f274410ea80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.150+0000 7f274b600640 1 --2- 172.21.15.103:0/2520305476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2744102780 0x7f27441067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.150+0000 7f274b600640 1 --2- 172.21.15.103:0/2520305476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744101dd0 0x7f27441021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.150+0000 7f274b600640 1 -- 172.21.15.103:0/2520305476 >> 172.21.15.103:0/2520305476 conn(0x7f27440fd8b0 msgr2=0x7f27440ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.150+0000 7f274b600640 1 -- 172.21.15.103:0/2520305476 shutdown_connections 2024-09-16T02:03:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.150+0000 7f274b600640 1 -- 172.21.15.103:0/2520305476 wait complete. 2024-09-16T02:03:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.151+0000 7f274b600640 1 Processor -- start 2024-09-16T02:03:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.152+0000 7f274b600640 1 -- start start 2024-09-16T02:03:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.152+0000 7f274b600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744101dd0 0x7f27440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.152+0000 7f274b600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2744102780 0x7f2744074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.152+0000 7f274b600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2744106f00 0x7f2744074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.152+0000 7f274b600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2744111fb0 con 0x7f2744106f00 2024-09-16T02:03:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.152+0000 7f274b600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2744111e30 con 0x7f2744102780 2024-09-16T02:03:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.152+0000 7f274b600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2744112130 con 0x7f2744101dd0 2024-09-16T02:03:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2749600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2744106f00 0x7f2744074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2743e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2744102780 0x7f2744074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2748c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744101dd0 0x7f27440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2749600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2744106f00 0x7f2744074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59092/0 (socket says 172.21.15.103:59092) 2024-09-16T02:03:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2743e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2744102780 0x7f2744074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:45826/0 (socket says 172.21.15.103:45826) 2024-09-16T02:03:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2749600640 1 -- 172.21.15.103:0/1908697974 learned_addr learned my addr 172.21.15.103:0/1908697974 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2743e00640 1 -- 172.21.15.103:0/1908697974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744101dd0 msgr2=0x7f27440744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2743e00640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744101dd0 0x7f27440744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2743e00640 1 -- 172.21.15.103:0/1908697974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2744106f00 msgr2=0x7f2744074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2743e00640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2744106f00 0x7f2744074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2743e00640 1 -- 172.21.15.103:0/1908697974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27441a6870 con 0x7f2744102780 2024-09-16T02:03:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2748c00640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744101dd0 0x7f27440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.153+0000 7f2743e00640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2744102780 0x7f2744074a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f273000e9f0 tx=0x7f273000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.154+0000 7f2741600640 1 -- 172.21.15.103:0/1908697974 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273000ccc0 con 0x7f2744102780 2024-09-16T02:03:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.154+0000 7f2741600640 1 -- 172.21.15.103:0/1908697974 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2730010910 con 0x7f2744102780 2024-09-16T02:03:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.154+0000 7f274b600640 1 -- 172.21.15.103:0/1908697974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27441a6b60 con 0x7f2744102780 2024-09-16T02:03:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.154+0000 7f2741600640 1 -- 172.21.15.103:0/1908697974 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730018960 con 0x7f2744102780 2024-09-16T02:03:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.154+0000 7f274b600640 1 -- 172.21.15.103:0/1908697974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27441a70a0 con 0x7f2744102780 2024-09-16T02:03:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.155+0000 7f274b600640 1 -- 172.21.15.103:0/1908697974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27441021b0 con 0x7f2744102780 2024-09-16T02:03:20.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.157+0000 7f2741600640 1 -- 172.21.15.103:0/1908697974 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2730010ac0 con 0x7f2744102780 2024-09-16T02:03:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.158+0000 7f2741600640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f272c0a3240 0x7f272c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.158+0000 7f2741600640 1 -- 172.21.15.103:0/1908697974 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f27300cf960 con 0x7f2744102780 2024-09-16T02:03:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.158+0000 7f2748c00640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f272c0a3240 0x7f272c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.159+0000 7f2748c00640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f272c0a3240 0x7f272c0a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2738009fc0 tx=0x7f2738005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.160+0000 7f2741600640 1 -- 172.21.15.103:0/1908697974 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f273009c430 con 0x7f2744102780 2024-09-16T02:03:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.377+0000 7f274b600640 1 -- 172.21.15.103:0/1908697974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2744075a20 con 0x7f2744102780 2024-09-16T02:03:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.379+0000 7f2741600640 1 -- 172.21.15.103:0/1908697974 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8890 (secure 0 0 0) 0x7f2744075a20 con 0x7f2744102780 2024-09-16T02:03:20.381 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:20.381 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":11,"btime":"2024-09-16T02:03:19:496608+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T02:03:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 11 2024-09-16T02:03:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 -- 172.21.15.103:0/1908697974 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f272c0a3240 msgr2=0x7f272c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f272c0a3240 0x7f272c0a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2738009fc0 tx=0x7f2738005f90 comp rx=0 tx=0).stop 2024-09-16T02:03:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 -- 172.21.15.103:0/1908697974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2744102780 msgr2=0x7f2744074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2744102780 0x7f2744074a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f273000e9f0 tx=0x7f273000eeb0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 -- 172.21.15.103:0/1908697974 shutdown_connections 2024-09-16T02:03:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f272c0a3240 0x7f272c0a5700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2744106f00 0x7f2744074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2744102780 0x7f2744074a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 --2- 172.21.15.103:0/1908697974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744101dd0 0x7f27440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 -- 172.21.15.103:0/1908697974 >> 172.21.15.103:0/1908697974 conn(0x7f27440fd8b0 msgr2=0x7f27440febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 -- 172.21.15.103:0/1908697974 shutdown_connections 2024-09-16T02:03:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:20.381+0000 7f274b600640 1 -- 172.21.15.103:0/1908697974 wait complete. 2024-09-16T02:03:20.723 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.103:6838/3043303715,v1:172.21.15.103:6839/3043303715] up:boot 2024-09-16T02:03:20.723 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: fsmap 10 up:standby 2024-09-16T02:03:20.723 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T02:03:20.724 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:20.724 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:20.724 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:20.724 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:20.724 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:20.724 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:20.724 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: Deploying daemon mds.a on smithi103 2024-09-16T02:03:20.724 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:20 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1908697974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.103:6838/3043303715,v1:172.21.15.103:6839/3043303715] up:boot 2024-09-16T02:03:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: fsmap 10 up:standby 2024-09-16T02:03:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T02:03:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:20.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: Deploying daemon mds.a on smithi103 2024-09-16T02:03:20.774 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:20 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1908697974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.103:6838/3043303715,v1:172.21.15.103:6839/3043303715] up:boot 2024-09-16T02:03:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: fsmap 10 up:standby 2024-09-16T02:03:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T02:03:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:20.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:20.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:20.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:20.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:20.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:20.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: Deploying daemon mds.a on smithi103 2024-09-16T02:03:20.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:20 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1908697974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:21.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:21 smithi103 ceph-mon[30398]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:22.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:21 smithi136 ceph-mon[35323]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:22.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:21 smithi172 ceph-mon[35340]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:22.483 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:03:22 smithi103 systemd[1]: Starting Ceph mds.a for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:03:22.734 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:03:22 smithi103 podman[71682]: 2024-09-16 02:03:22.48174621 +0000 UTC m=+0.102314997 container create ddd596b8db3cf4df00af744da834beb82d7d2a9ff54ca2bef02525617bdc813e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-16T02:03:22.734 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:03:22 smithi103 podman[71682]: 2024-09-16 02:03:22.406774944 +0000 UTC m=+0.027343738 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:03:22.734 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:03:22 smithi103 podman[71682]: 2024-09-16 02:03:22.726344402 +0000 UTC m=+0.346913211 container init ddd596b8db3cf4df00af744da834beb82d7d2a9ff54ca2bef02525617bdc813e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:03:22.734 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:03:22 smithi103 podman[71682]: 2024-09-16 02:03:22.732825636 +0000 UTC m=+0.353394426 container start ddd596b8db3cf4df00af744da834beb82d7d2a9ff54ca2bef02525617bdc813e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T02:03:23.005 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:03:22 smithi103 ceph-mds[71701]: -- 172.21.15.103:0/1795860106 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b037cf860 con 0x557b037ac800 2024-09-16T02:03:23.005 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:03:22 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-a[71697]: starting mds.a at 2024-09-16T02:03:23.005 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:03:22 smithi103 bash[71682]: ddd596b8db3cf4df00af744da834beb82d7d2a9ff54ca2bef02525617bdc813e 2024-09-16T02:03:23.005 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:03:22 smithi103 systemd[1]: Started Ceph mds.a for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:03:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:23 smithi103 ceph-mon[30398]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:24.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:23 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:23 smithi136 ceph-mon[35323]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:24.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:24.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:24.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:24.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:24.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:23 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:23 smithi172 ceph-mon[35340]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T02:03:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T02:03:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:23 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:24.070 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:03:24.454 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:24.789 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:24 smithi136 ceph-mon[35323]: Deploying daemon mds.b on smithi136 2024-09-16T02:03:24.789 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:24 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.103:6840/2552214171,v1:172.21.15.103:6841/2552214171] up:boot 2024-09-16T02:03:24.789 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:24 smithi136 ceph-mon[35323]: fsmap 11 up:standby 2024-09-16T02:03:24.789 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:24 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T02:03:24.856 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:24 smithi103 ceph-mon[30398]: Deploying daemon mds.b on smithi136 2024-09-16T02:03:24.857 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:24 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.103:6840/2552214171,v1:172.21.15.103:6841/2552214171] up:boot 2024-09-16T02:03:24.857 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:24 smithi103 ceph-mon[30398]: fsmap 11 up:standby 2024-09-16T02:03:24.857 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:24 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T02:03:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.966+0000 7f8f35200640 1 -- 172.21.15.103:0/3055495977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f301047e0 msgr2=0x7f8f30108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.966+0000 7f8f35200640 1 --2- 172.21.15.103:0/3055495977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f301047e0 0x7f8f30108830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009900 tx=0x7f8f1c02a410 comp rx=0 tx=0).stop 2024-09-16T02:03:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.966+0000 7f8f35200640 1 -- 172.21.15.103:0/3055495977 shutdown_connections 2024-09-16T02:03:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.966+0000 7f8f35200640 1 --2- 172.21.15.103:0/3055495977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f30108f60 0x7f8f30110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.966+0000 7f8f35200640 1 --2- 172.21.15.103:0/3055495977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f301047e0 0x7f8f30108830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.966+0000 7f8f35200640 1 --2- 172.21.15.103:0/3055495977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f30103e30 0x7f8f30104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.966+0000 7f8f35200640 1 -- 172.21.15.103:0/3055495977 >> 172.21.15.103:0/3055495977 conn(0x7f8f300ff8d0 msgr2=0x7f8f30101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.966+0000 7f8f35200640 1 -- 172.21.15.103:0/3055495977 shutdown_connections 2024-09-16T02:03:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.966+0000 7f8f35200640 1 -- 172.21.15.103:0/3055495977 wait complete. 2024-09-16T02:03:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.967+0000 7f8f35200640 1 Processor -- start 2024-09-16T02:03:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.967+0000 7f8f35200640 1 -- start start 2024-09-16T02:03:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f35200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f30103e30 0x7f8f3019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f35200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f301047e0 0x7f8f3019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f35200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f30108f60 0x7f8f3019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f35200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301140b0 con 0x7f8f301047e0 2024-09-16T02:03:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f35200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30113f30 con 0x7f8f30108f60 2024-09-16T02:03:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f35200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30114230 con 0x7f8f30103e30 2024-09-16T02:03:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f2f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f30103e30 0x7f8f3019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f2ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f301047e0 0x7f8f3019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f2f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f30103e30 0x7f8f3019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:46452/0 (socket says 172.21.15.103:46452) 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f2f400640 1 -- 172.21.15.103:0/1884364461 learned_addr learned my addr 172.21.15.103:0/1884364461 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.968+0000 7f8f2fe00640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f30108f60 0x7f8f3019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f2ea00640 1 -- 172.21.15.103:0/1884364461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f30103e30 msgr2=0x7f8f3019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f2ea00640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f30103e30 0x7f8f3019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f2ea00640 1 -- 172.21.15.103:0/1884364461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f30108f60 msgr2=0x7f8f3019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f2ea00640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f30108f60 0x7f8f3019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f2ea00640 1 -- 172.21.15.103:0/1884364461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3019f580 con 0x7f8f301047e0 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f2ea00640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f301047e0 0x7f8f3019ab10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c02a930 tx=0x7f8f1c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f27e00640 1 -- 172.21.15.103:0/1884364461 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c03f070 con 0x7f8f301047e0 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f27e00640 1 -- 172.21.15.103:0/1884364461 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1c03b460 con 0x7f8f301047e0 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f27e00640 1 -- 172.21.15.103:0/1884364461 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c035030 con 0x7f8f301047e0 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f35200640 1 -- 172.21.15.103:0/1884364461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3019f810 con 0x7f8f301047e0 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.969+0000 7f8f35200640 1 -- 172.21.15.103:0/1884364461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f301a7150 con 0x7f8f301047e0 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.970+0000 7f8f35200640 1 -- 172.21.15.103:0/1884364461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f30104290 con 0x7f8f301047e0 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.972+0000 7f8f27e00640 1 -- 172.21.15.103:0/1884364461 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8f1c042070 con 0x7f8f301047e0 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.973+0000 7f8f27e00640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8f140a32c0 0x7f8f140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.974+0000 7f8f27e00640 1 -- 172.21.15.103:0/1884364461 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f8f1c0efba0 con 0x7f8f301047e0 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.974+0000 7f8f2f400640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8f140a32c0 0x7f8f140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.974+0000 7f8f2f400640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8f140a32c0 0x7f8f140a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000e250 tx=0x7f8f2000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:24.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:24.978+0000 7f8f27e00640 1 -- 172.21.15.103:0/1884364461 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f1c0bc670 con 0x7f8f301047e0 2024-09-16T02:03:25.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:24 smithi172 ceph-mon[35340]: Deploying daemon mds.b on smithi136 2024-09-16T02:03:25.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:24 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.103:6840/2552214171,v1:172.21.15.103:6841/2552214171] up:boot 2024-09-16T02:03:25.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:24 smithi172 ceph-mon[35340]: fsmap 11 up:standby 2024-09-16T02:03:25.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:24 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T02:03:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.188+0000 7f8f35200640 1 -- 172.21.15.103:0/1884364461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f3019b930 con 0x7f8f301047e0 2024-09-16T02:03:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.188+0000 7f8f27e00640 1 -- 172.21.15.103:0/1884364461 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9719 (secure 0 0 0) 0x7f8f1c0c0970 con 0x7f8f301047e0 2024-09-16T02:03:25.191 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:25.191 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":12,"btime":"2024-09-16T02:03:23:593940+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T02:03:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 12 2024-09-16T02:03:25.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 -- 172.21.15.103:0/1884364461 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8f140a32c0 msgr2=0x7f8f140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:25.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8f140a32c0 0x7f8f140a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000e250 tx=0x7f8f2000b040 comp rx=0 tx=0).stop 2024-09-16T02:03:25.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 -- 172.21.15.103:0/1884364461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f301047e0 msgr2=0x7f8f3019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:25.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f301047e0 0x7f8f3019ab10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c02a930 tx=0x7f8f1c003c00 comp rx=0 tx=0).stop 2024-09-16T02:03:25.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 -- 172.21.15.103:0/1884364461 shutdown_connections 2024-09-16T02:03:25.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8f140a32c0 0x7f8f140a5780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f30108f60 0x7f8f3019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f301047e0 0x7f8f3019ab10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 --2- 172.21.15.103:0/1884364461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f30103e30 0x7f8f3019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 -- 172.21.15.103:0/1884364461 >> 172.21.15.103:0/1884364461 conn(0x7f8f300ff8d0 msgr2=0x7f8f30100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 -- 172.21.15.103:0/1884364461 shutdown_connections 2024-09-16T02:03:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:25.191+0000 7f8f35200640 1 -- 172.21.15.103:0/1884364461 wait complete. 2024-09-16T02:03:25.523 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:03:25 smithi136 systemd[1]: Started Ceph mds.b for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:03:25.886 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:25 smithi103 ceph-mon[30398]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:25.886 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:25 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1884364461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:25.886 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:25.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:25.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:25.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:25.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:25.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:25 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:03:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:25 smithi136 ceph-mon[35323]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:25 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1884364461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:26.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:26.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:26.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:25 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:03:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:25 smithi172 ceph-mon[35340]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:25 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1884364461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:25 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:03:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:26 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] up:boot 2024-09-16T02:03:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:26 smithi172 ceph-mon[35340]: fsmap 12 up:standby 2024-09-16T02:03:26.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:26 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:03:26.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:26 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] up:boot 2024-09-16T02:03:26.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:26 smithi136 ceph-mon[35323]: fsmap 12 up:standby 2024-09-16T02:03:26.876 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:26 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:03:27.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:26 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] up:boot 2024-09-16T02:03:27.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:26 smithi103 ceph-mon[30398]: fsmap 12 up:standby 2024-09-16T02:03:27.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:26 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:03:27.926 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:03:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:27 smithi103 ceph-mon[30398]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:03:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:27 smithi136 ceph-mon[35323]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:03:28.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:27 smithi172 ceph-mon[35340]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:03:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:03:28.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:28.954 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:03:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:03:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:29.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.500+0000 7f6cafe00640 1 -- 172.21.15.103:0/1333358803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb0102b90 msgr2=0x7f6cb0102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:29.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.500+0000 7f6cafe00640 1 --2- 172.21.15.103:0/1333358803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb0102b90 0x7f6cb0102f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009900 tx=0x7f6ca002a480 comp rx=0 tx=0).stop 2024-09-16T02:03:29.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.500+0000 7f6cafe00640 1 -- 172.21.15.103:0/1333358803 shutdown_connections 2024-09-16T02:03:29.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.500+0000 7f6cafe00640 1 --2- 172.21.15.103:0/1333358803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cb010bfc0 0x7f6cb010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:29.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.500+0000 7f6cafe00640 1 --2- 172.21.15.103:0/1333358803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cb01034b0 0x7f6cb010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:29.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.500+0000 7f6cafe00640 1 --2- 172.21.15.103:0/1333358803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb0102b90 0x7f6cb0102f70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:29.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.500+0000 7f6cafe00640 1 -- 172.21.15.103:0/1333358803 >> 172.21.15.103:0/1333358803 conn(0x7f6cb00fc610 msgr2=0x7f6cb00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:29.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.500+0000 7f6cafe00640 1 -- 172.21.15.103:0/1333358803 shutdown_connections 2024-09-16T02:03:29.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.500+0000 7f6cafe00640 1 -- 172.21.15.103:0/1333358803 wait complete. 2024-09-16T02:03:29.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.501+0000 7f6cafe00640 1 Processor -- start 2024-09-16T02:03:29.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.501+0000 7f6cafe00640 1 -- start start 2024-09-16T02:03:29.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6cafe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb0102b90 0x7f6cb019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:29.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6cafe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cb01034b0 0x7f6cb019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:29.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6cafe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cb010bfc0 0x7f6cb019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:29.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6cafe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0111e10 con 0x7f6cb010bfc0 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6cafe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0111c90 con 0x7f6cb0102b90 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6cafe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0111f90 con 0x7f6cb01034b0 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6ca6000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cb01034b0 0x7f6cb019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6caf400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cb010bfc0 0x7f6cb019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6ca6000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cb01034b0 0x7f6cb019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:36248/0 (socket says 172.21.15.103:36248) 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6ca6000640 1 -- 172.21.15.103:0/2500470320 learned_addr learned my addr 172.21.15.103:0/2500470320 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6caea00640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb0102b90 0x7f6cb019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6ca6000640 1 -- 172.21.15.103:0/2500470320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb0102b90 msgr2=0x7f6cb019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6ca6000640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb0102b90 0x7f6cb019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6ca6000640 1 -- 172.21.15.103:0/2500470320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cb010bfc0 msgr2=0x7f6cb019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6ca6000640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cb010bfc0 0x7f6cb019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.502+0000 7f6ca6000640 1 -- 172.21.15.103:0/2500470320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb019f4d0 con 0x7f6cb01034b0 2024-09-16T02:03:29.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.503+0000 7f6ca6000640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cb01034b0 0x7f6cb019aa60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800e9f0 tx=0x7f6c9800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:29.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.503+0000 7f6ca7e00640 1 -- 172.21.15.103:0/2500470320 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800cac0 con 0x7f6cb01034b0 2024-09-16T02:03:29.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.503+0000 7f6ca7e00640 1 -- 172.21.15.103:0/2500470320 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c98005500 con 0x7f6cb01034b0 2024-09-16T02:03:29.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.503+0000 7f6ca7e00640 1 -- 172.21.15.103:0/2500470320 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98010430 con 0x7f6cb01034b0 2024-09-16T02:03:29.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.503+0000 7f6cafe00640 1 -- 172.21.15.103:0/2500470320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb019f7c0 con 0x7f6cb01034b0 2024-09-16T02:03:29.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.503+0000 7f6cafe00640 1 -- 172.21.15.103:0/2500470320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb01a70a0 con 0x7f6cb01034b0 2024-09-16T02:03:29.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.504+0000 7f6cafe00640 1 -- 172.21.15.103:0/2500470320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb0102f70 con 0x7f6cb01034b0 2024-09-16T02:03:29.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.507+0000 7f6ca7e00640 1 -- 172.21.15.103:0/2500470320 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6c98004f70 con 0x7f6cb01034b0 2024-09-16T02:03:29.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.508+0000 7f6ca7e00640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c940a32c0 0x7f6c940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:29.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.509+0000 7f6ca7e00640 1 -- 172.21.15.103:0/2500470320 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f6c980cf9c0 con 0x7f6cb01034b0 2024-09-16T02:03:29.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.509+0000 7f6caea00640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c940a32c0 0x7f6c940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:29.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.509+0000 7f6caea00640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c940a32c0 0x7f6c940a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0000c00 tx=0x7f6ca0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:29.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.509+0000 7f6ca7e00640 1 -- 172.21.15.103:0/2500470320 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c980541b0 con 0x7f6cb01034b0 2024-09-16T02:03:29.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.742+0000 7f6cafe00640 1 -- 172.21.15.103:0/2500470320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cb019b880 con 0x7f6cb01034b0 2024-09-16T02:03:29.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.743+0000 7f6ca7e00640 1 -- 172.21.15.103:0/2500470320 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10548 (secure 0 0 0) 0x7f6cb019b880 con 0x7f6cb01034b0 2024-09-16T02:03:29.745 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:29 smithi103 ceph-mon[30398]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:29.745 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:29.746 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:29.746 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:29.746 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:29.746 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:29.746 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":13,"btime":"2024-09-16T02:03:25:606271+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-16T02:03:29.747 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 13 2024-09-16T02:03:29.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 -- 172.21.15.103:0/2500470320 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c940a32c0 msgr2=0x7f6c940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:29.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c940a32c0 0x7f6c940a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0000c00 tx=0x7f6ca0005b50 comp rx=0 tx=0).stop 2024-09-16T02:03:29.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 -- 172.21.15.103:0/2500470320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cb01034b0 msgr2=0x7f6cb019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:29.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cb01034b0 0x7f6cb019aa60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800e9f0 tx=0x7f6c9800edf0 comp rx=0 tx=0).stop 2024-09-16T02:03:29.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 -- 172.21.15.103:0/2500470320 shutdown_connections 2024-09-16T02:03:29.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6c940a32c0 0x7f6c940a5780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:29.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cb010bfc0 0x7f6cb019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:29.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cb01034b0 0x7f6cb019aa60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:29.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 --2- 172.21.15.103:0/2500470320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb0102b90 0x7f6cb019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:29.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 -- 172.21.15.103:0/2500470320 >> 172.21.15.103:0/2500470320 conn(0x7f6cb00fc610 msgr2=0x7f6cb010d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:29.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 -- 172.21.15.103:0/2500470320 shutdown_connections 2024-09-16T02:03:29.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:29.746+0000 7f6cafe00640 1 -- 172.21.15.103:0/2500470320 wait complete. 2024-09-16T02:03:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:29 smithi136 ceph-mon[35323]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:30.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:29 smithi172 ceph-mon[35340]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:30.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:30.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:30.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:30.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:30.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-16T02:03:30.725 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:30.985 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:30 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2500470320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:30.985 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:30.985 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:30.985 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:30.985 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:03:30.985 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:30 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2500470320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:03:31.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:30 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2500470320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:03:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:03:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:03:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:03:31.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.213+0000 7f23f4000640 1 -- 172.21.15.103:0/4001848050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23ec074590 msgr2=0x7f23ec074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:31.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.213+0000 7f23f4000640 1 --2- 172.21.15.103:0/4001848050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23ec074590 0x7f23ec074970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009900 tx=0x7f23e002a480 comp rx=0 tx=0).stop 2024-09-16T02:03:31.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.213+0000 7f23f4000640 1 -- 172.21.15.103:0/4001848050 shutdown_connections 2024-09-16T02:03:31.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.213+0000 7f23f4000640 1 --2- 172.21.15.103:0/4001848050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23ec112b90 0x7f23ec114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.213+0000 7f23f4000640 1 --2- 172.21.15.103:0/4001848050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23ec074eb0 0x7f23ec112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.213+0000 7f23f4000640 1 --2- 172.21.15.103:0/4001848050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23ec074590 0x7f23ec074970 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.213+0000 7f23f4000640 1 -- 172.21.15.103:0/4001848050 >> 172.21.15.103:0/4001848050 conn(0x7f23ec0fd8c0 msgr2=0x7f23ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.213+0000 7f23f4000640 1 -- 172.21.15.103:0/4001848050 shutdown_connections 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.213+0000 7f23f4000640 1 -- 172.21.15.103:0/4001848050 wait complete. 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f4000640 1 Processor -- start 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f4000640 1 -- start start 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f4000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23ec074590 0x7f23ec19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f4000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23ec074eb0 0x7f23ec19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f4000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23ec112b90 0x7f23ec1a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:31.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f4000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1185c0 con 0x7f23ec074590 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f4000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec118440 con 0x7f23ec074eb0 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f4000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec118740 con 0x7f23ec112b90 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23ec074590 0x7f23ec19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23ec074590 0x7f23ec19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45656/0 (socket says 172.21.15.103:45656) 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.215+0000 7f23f1600640 1 -- 172.21.15.103:0/2214841266 learned_addr learned my addr 172.21.15.103:0/2214841266 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.216+0000 7f23f2000640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23ec112b90 0x7f23ec1a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.216+0000 7f23f0c00640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23ec074eb0 0x7f23ec19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.216+0000 7f23f1600640 1 -- 172.21.15.103:0/2214841266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23ec112b90 msgr2=0x7f23ec1a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.216+0000 7f23f1600640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23ec112b90 0x7f23ec1a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.216+0000 7f23f1600640 1 -- 172.21.15.103:0/2214841266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23ec074eb0 msgr2=0x7f23ec19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.216+0000 7f23f1600640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23ec074eb0 0x7f23ec19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:31.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.216+0000 7f23f1600640 1 -- 172.21.15.103:0/2214841266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec1a3b80 con 0x7f23ec074590 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.216+0000 7f23f1600640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23ec074590 0x7f23ec19eb50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f23e0004770 tx=0x7f23e0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.216+0000 7f23ea000640 1 -- 172.21.15.103:0/2214841266 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e003f070 con 0x7f23ec074590 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.216+0000 7f23ea000640 1 -- 172.21.15.103:0/2214841266 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e0031ca0 con 0x7f23ec074590 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.217+0000 7f23f4000640 1 -- 172.21.15.103:0/2214841266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ec1a3e10 con 0x7f23ec074590 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.217+0000 7f23ea000640 1 -- 172.21.15.103:0/2214841266 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0035030 con 0x7f23ec074590 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.217+0000 7f23f4000640 1 -- 172.21.15.103:0/2214841266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ec1ab700 con 0x7f23ec074590 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.218+0000 7f23f4000640 1 -- 172.21.15.103:0/2214841266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ec074970 con 0x7f23ec074590 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.219+0000 7f23ea000640 1 -- 172.21.15.103:0/2214841266 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f23e0042070 con 0x7f23ec074590 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.221+0000 7f23ea000640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23d40a32c0 0x7f23d40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.221+0000 7f23ea000640 1 -- 172.21.15.103:0/2214841266 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f23e00efbc0 con 0x7f23ec074590 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.221+0000 7f23f0c00640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23d40a32c0 0x7f23d40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:31.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.221+0000 7f23f0c00640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23d40a32c0 0x7f23d40a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0fecf0 tx=0x7f23d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:31.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.225+0000 7f23ea000640 1 -- 172.21.15.103:0/2214841266 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23e00bcce0 con 0x7f23ec074590 2024-09-16T02:03:31.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.402+0000 7f23f4000640 1 -- 172.21.15.103:0/2214841266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f23ec19feb0 con 0x7f23ec074590 2024-09-16T02:03:31.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.402+0000 7f23ea000640 1 -- 172.21.15.103:0/2214841266 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f23e00bc5e0 con 0x7f23ec074590 2024-09-16T02:03:31.405 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:31.405 INFO:teuthology.orchestra.run.smithi103.stdout:[ 2024-09-16T02:03:31.405 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:03:31.405 INFO:teuthology.orchestra.run.smithi103.stdout: "poolnum": 1, 2024-09-16T02:03:31.405 INFO:teuthology.orchestra.run.smithi103.stdout: "poolname": ".mgr" 2024-09-16T02:03:31.405 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:03:31.405 INFO:teuthology.orchestra.run.smithi103.stdout:] 2024-09-16T02:03:31.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 -- 172.21.15.103:0/2214841266 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23d40a32c0 msgr2=0x7f23d40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:31.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23d40a32c0 0x7f23d40a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0fecf0 tx=0x7f23d8005f90 comp rx=0 tx=0).stop 2024-09-16T02:03:31.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 -- 172.21.15.103:0/2214841266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23ec074590 msgr2=0x7f23ec19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:31.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23ec074590 0x7f23ec19eb50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f23e0004770 tx=0x7f23e0003c10 comp rx=0 tx=0).stop 2024-09-16T02:03:31.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 -- 172.21.15.103:0/2214841266 shutdown_connections 2024-09-16T02:03:31.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23d40a32c0 0x7f23d40a5780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:31.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23ec112b90 0x7f23ec1a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:31.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23ec074eb0 0x7f23ec19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:31.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 --2- 172.21.15.103:0/2214841266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23ec074590 0x7f23ec19eb50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:31.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 -- 172.21.15.103:0/2214841266 >> 172.21.15.103:0/2214841266 conn(0x7f23ec0fd8c0 msgr2=0x7f23ec0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:31.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 -- 172.21.15.103:0/2214841266 shutdown_connections 2024-09-16T02:03:31.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:31.405+0000 7f23f4000640 1 -- 172.21.15.103:0/2214841266 wait complete. 2024-09-16T02:03:32.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:31 smithi103 ceph-mon[30398]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:32.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:31 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2214841266' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T02:03:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:31 smithi136 ceph-mon[35323]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:31 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2214841266' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T02:03:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:31 smithi172 ceph-mon[35340]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:31 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2214841266' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T02:03:32.080 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T02:03:32.080 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T02:03:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:32.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.975+0000 7f992f200640 1 -- 172.21.15.103:0/1658447865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9930103e30 msgr2=0x7f9930104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.975+0000 7f992f200640 1 --2- 172.21.15.103:0/1658447865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9930103e30 0x7f9930104210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9920009900 tx=0x7f992002a410 comp rx=0 tx=0).stop 2024-09-16T02:03:32.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.976+0000 7f992f200640 1 -- 172.21.15.103:0/1658447865 shutdown_connections 2024-09-16T02:03:32.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.976+0000 7f992f200640 1 --2- 172.21.15.103:0/1658447865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9930108f60 0x7f9930110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.976+0000 7f992f200640 1 --2- 172.21.15.103:0/1658447865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99301047e0 0x7f9930108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.976+0000 7f992f200640 1 --2- 172.21.15.103:0/1658447865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9930103e30 0x7f9930104210 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.976+0000 7f992f200640 1 -- 172.21.15.103:0/1658447865 >> 172.21.15.103:0/1658447865 conn(0x7f99300ff8d0 msgr2=0x7f9930101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:32.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.976+0000 7f992f200640 1 -- 172.21.15.103:0/1658447865 shutdown_connections 2024-09-16T02:03:32.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.976+0000 7f992f200640 1 -- 172.21.15.103:0/1658447865 wait complete. 2024-09-16T02:03:32.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992f200640 1 Processor -- start 2024-09-16T02:03:32.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992f200640 1 -- start start 2024-09-16T02:03:32.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9930103e30 0x7f993019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99301047e0 0x7f993019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9930108f60 0x7f993019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9930114010 con 0x7f9930103e30 2024-09-16T02:03:32.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9930113e90 con 0x7f99301047e0 2024-09-16T02:03:32.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9930114190 con 0x7f9930108f60 2024-09-16T02:03:32.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9930103e30 0x7f993019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9930103e30 0x7f993019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45676/0 (socket says 172.21.15.103:45676) 2024-09-16T02:03:32.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.977+0000 7f992de00640 1 -- 172.21.15.103:0/140298169 learned_addr learned my addr 172.21.15.103:0/140298169 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:32.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.978+0000 7f992d400640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99301047e0 0x7f993019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.978+0000 7f992e800640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9930108f60 0x7f993019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.978+0000 7f992de00640 1 -- 172.21.15.103:0/140298169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9930108f60 msgr2=0x7f993019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.978+0000 7f992de00640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9930108f60 0x7f993019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.978+0000 7f992de00640 1 -- 172.21.15.103:0/140298169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99301047e0 msgr2=0x7f993019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:32.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.978+0000 7f992de00640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99301047e0 0x7f993019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:32.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.978+0000 7f992de00640 1 -- 172.21.15.103:0/140298169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993019f460 con 0x7f9930103e30 2024-09-16T02:03:32.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.978+0000 7f992de00640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9930103e30 0x7f993019a4b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9920004770 tx=0x7f9920003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:32.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.978+0000 7f9926a00640 1 -- 172.21.15.103:0/140298169 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992003f070 con 0x7f9930103e30 2024-09-16T02:03:32.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.979+0000 7f9926a00640 1 -- 172.21.15.103:0/140298169 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9920031ca0 con 0x7f9930103e30 2024-09-16T02:03:32.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.979+0000 7f992f200640 1 -- 172.21.15.103:0/140298169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993019f6f0 con 0x7f9930103e30 2024-09-16T02:03:32.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.979+0000 7f9926a00640 1 -- 172.21.15.103:0/140298169 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920035030 con 0x7f9930103e30 2024-09-16T02:03:32.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.979+0000 7f992f200640 1 -- 172.21.15.103:0/140298169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99301a7030 con 0x7f9930103e30 2024-09-16T02:03:32.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.980+0000 7f992f200640 1 -- 172.21.15.103:0/140298169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9930105310 con 0x7f9930103e30 2024-09-16T02:03:32.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.982+0000 7f9926a00640 1 -- 172.21.15.103:0/140298169 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9920042070 con 0x7f9930103e30 2024-09-16T02:03:32.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.983+0000 7f9926a00640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99140a3240 0x7f99140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:32.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.983+0000 7f9926a00640 1 -- 172.21.15.103:0/140298169 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f99200efbd0 con 0x7f9930103e30 2024-09-16T02:03:32.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.983+0000 7f992d400640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99140a3240 0x7f99140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:32.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.984+0000 7f992d400640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99140a3240 0x7f99140a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f99300ffcf0 tx=0x7f991c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:32.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:32.987+0000 7f9926a00640 1 -- 172.21.15.103:0/140298169 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99200bcdb0 con 0x7f9930103e30 2024-09-16T02:03:33.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.162+0000 7f992f200640 1 -- 172.21.15.103:0/140298169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f993010c140 con 0x7f9930103e30 2024-09-16T02:03:33.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.713+0000 7f9926a00640 1 -- 172.21.15.103:0/140298169 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f99200bc6b0 con 0x7f9930103e30 2024-09-16T02:03:33.715 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' created 2024-09-16T02:03:33.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 -- 172.21.15.103:0/140298169 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99140a3240 msgr2=0x7f99140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:33.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99140a3240 0x7f99140a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f99300ffcf0 tx=0x7f991c005f90 comp rx=0 tx=0).stop 2024-09-16T02:03:33.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 -- 172.21.15.103:0/140298169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9930103e30 msgr2=0x7f993019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:33.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9930103e30 0x7f993019a4b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9920004770 tx=0x7f9920003c10 comp rx=0 tx=0).stop 2024-09-16T02:03:33.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 -- 172.21.15.103:0/140298169 shutdown_connections 2024-09-16T02:03:33.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99140a3240 0x7f99140a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:33.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9930108f60 0x7f993019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:33.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99301047e0 0x7f993019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:33.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 --2- 172.21.15.103:0/140298169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9930103e30 0x7f993019a4b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:33.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 -- 172.21.15.103:0/140298169 >> 172.21.15.103:0/140298169 conn(0x7f99300ff8d0 msgr2=0x7f9930100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:33.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.716+0000 7f992f200640 1 -- 172.21.15.103:0/140298169 shutdown_connections 2024-09-16T02:03:33.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:33.717+0000 7f992f200640 1 -- 172.21.15.103:0/140298169 wait complete. 2024-09-16T02:03:34.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:33 smithi103 ceph-mon[30398]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:34.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:33 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/140298169' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T02:03:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:33 smithi136 ceph-mon[35323]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:33 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/140298169' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T02:03:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:33 smithi172 ceph-mon[35340]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:33 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/140298169' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T02:03:34.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T02:03:34.719 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:35.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:34 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/140298169' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T02:03:35.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:34 smithi103 ceph-mon[30398]: osdmap e73: 12 total, 12 up, 12 in 2024-09-16T02:03:35.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:34 smithi103 ceph-mon[30398]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:34 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/140298169' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T02:03:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:34 smithi136 ceph-mon[35323]: osdmap e73: 12 total, 12 up, 12 in 2024-09-16T02:03:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:34 smithi136 ceph-mon[35323]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:34 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/140298169' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T02:03:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:34 smithi172 ceph-mon[35340]: osdmap e73: 12 total, 12 up, 12 in 2024-09-16T02:03:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:34 smithi172 ceph-mon[35340]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:35.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.319+0000 7fc635200640 1 -- 172.21.15.103:0/259100874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6301049a0 msgr2=0x7fc6301089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:35.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.319+0000 7fc635200640 1 --2- 172.21.15.103:0/259100874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6301049a0 0x7fc6301089f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009900 tx=0x7fc61c02a410 comp rx=0 tx=0).stop 2024-09-16T02:03:35.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.319+0000 7fc635200640 1 -- 172.21.15.103:0/259100874 shutdown_connections 2024-09-16T02:03:35.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.319+0000 7fc635200640 1 --2- 172.21.15.103:0/259100874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc630109120 0x7fc630110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:35.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.319+0000 7fc635200640 1 --2- 172.21.15.103:0/259100874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6301049a0 0x7fc6301089f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:35.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.319+0000 7fc635200640 1 --2- 172.21.15.103:0/259100874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc630103ff0 0x7fc6301043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:35.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.319+0000 7fc635200640 1 -- 172.21.15.103:0/259100874 >> 172.21.15.103:0/259100874 conn(0x7fc6300ffa90 msgr2=0x7fc630101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:35.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.319+0000 7fc635200640 1 -- 172.21.15.103:0/259100874 shutdown_connections 2024-09-16T02:03:35.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.319+0000 7fc635200640 1 -- 172.21.15.103:0/259100874 wait complete. 2024-09-16T02:03:35.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.320+0000 7fc635200640 1 Processor -- start 2024-09-16T02:03:35.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc635200640 1 -- start start 2024-09-16T02:03:35.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc635200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc630103ff0 0x7fc6301a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc635200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6301049a0 0x7fc6301a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc635200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc630109120 0x7fc63019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc635200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc630114220 con 0x7fc6301049a0 2024-09-16T02:03:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc635200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301140a0 con 0x7fc630103ff0 2024-09-16T02:03:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc635200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301143a0 con 0x7fc630109120 2024-09-16T02:03:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc62d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6301049a0 0x7fc6301a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc62de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc630103ff0 0x7fc6301a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc62de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc630103ff0 0x7fc6301a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:37236/0 (socket says 172.21.15.103:37236) 2024-09-16T02:03:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc62d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6301049a0 0x7fc6301a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45702/0 (socket says 172.21.15.103:45702) 2024-09-16T02:03:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc62de00640 1 -- 172.21.15.103:0/1720800546 learned_addr learned my addr 172.21.15.103:0/1720800546 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:35.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.321+0000 7fc62e800640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc630109120 0x7fc63019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:35.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.322+0000 7fc62d400640 1 -- 172.21.15.103:0/1720800546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc630109120 msgr2=0x7fc63019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.322+0000 7fc62d400640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc630109120 0x7fc63019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.322+0000 7fc62d400640 1 -- 172.21.15.103:0/1720800546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc630103ff0 msgr2=0x7fc6301a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.322+0000 7fc62d400640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc630103ff0 0x7fc6301a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.322+0000 7fc62d400640 1 -- 172.21.15.103:0/1720800546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63019acf0 con 0x7fc6301049a0 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.322+0000 7fc62d400640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6301049a0 0x7fc6301a0940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc61c006010 tx=0x7fc61c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.322+0000 7fc626a00640 1 -- 172.21.15.103:0/1720800546 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c03f030 con 0x7fc6301049a0 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.322+0000 7fc626a00640 1 -- 172.21.15.103:0/1720800546 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c003ce0 con 0x7fc6301049a0 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.322+0000 7fc626a00640 1 -- 172.21.15.103:0/1720800546 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c035070 con 0x7fc6301049a0 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.322+0000 7fc635200640 1 -- 172.21.15.103:0/1720800546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63019af80 con 0x7fc6301049a0 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.323+0000 7fc635200640 1 -- 172.21.15.103:0/1720800546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6301a72d0 con 0x7fc6301049a0 2024-09-16T02:03:35.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.324+0000 7fc635200640 1 -- 172.21.15.103:0/1720800546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc630104450 con 0x7fc6301049a0 2024-09-16T02:03:35.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.326+0000 7fc626a00640 1 -- 172.21.15.103:0/1720800546 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc61c042030 con 0x7fc6301049a0 2024-09-16T02:03:35.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.327+0000 7fc626a00640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc6140a3240 0x7fc6140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:35.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.327+0000 7fc626a00640 1 -- 172.21.15.103:0/1720800546 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7fc61c0efaf0 con 0x7fc6301049a0 2024-09-16T02:03:35.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.327+0000 7fc62de00640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc6140a3240 0x7fc6140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:35.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.328+0000 7fc62de00640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc6140a3240 0x7fc6140a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc620009a40 tx=0x7fc6200091c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:35.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.331+0000 7fc626a00640 1 -- 172.21.15.103:0/1720800546 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc61c0bc450 con 0x7fc6301049a0 2024-09-16T02:03:35.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:35.506+0000 7fc635200640 1 -- 172.21.15.103:0/1720800546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc63010c300 con 0x7fc6301049a0 2024-09-16T02:03:36.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.021+0000 7fc626a00640 1 -- 172.21.15.103:0/1720800546 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7fc61c0c0750 con 0x7fc6301049a0 2024-09-16T02:03:36.023 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' created 2024-09-16T02:03:36.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.024+0000 7fc635200640 1 -- 172.21.15.103:0/1720800546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc6140a3240 msgr2=0x7fc6140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:36.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.024+0000 7fc635200640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc6140a3240 0x7fc6140a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc620009a40 tx=0x7fc6200091c0 comp rx=0 tx=0).stop 2024-09-16T02:03:36.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.024+0000 7fc635200640 1 -- 172.21.15.103:0/1720800546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6301049a0 msgr2=0x7fc6301a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:36.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.024+0000 7fc635200640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6301049a0 0x7fc6301a0940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc61c006010 tx=0x7fc61c0042f0 comp rx=0 tx=0).stop 2024-09-16T02:03:36.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.025+0000 7fc635200640 1 -- 172.21.15.103:0/1720800546 shutdown_connections 2024-09-16T02:03:36.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.025+0000 7fc635200640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc6140a3240 0x7fc6140a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:36.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.025+0000 7fc635200640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc630109120 0x7fc63019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:36.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.025+0000 7fc635200640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6301049a0 0x7fc6301a0940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:36.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.025+0000 7fc635200640 1 --2- 172.21.15.103:0/1720800546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc630103ff0 0x7fc6301a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:36.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.025+0000 7fc635200640 1 -- 172.21.15.103:0/1720800546 >> 172.21.15.103:0/1720800546 conn(0x7fc6300ffa90 msgr2=0x7fc6301009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:36.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.025+0000 7fc635200640 1 -- 172.21.15.103:0/1720800546 shutdown_connections 2024-09-16T02:03:36.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:36.025+0000 7fc635200640 1 -- 172.21.15.103:0/1720800546 wait complete. 2024-09-16T02:03:36.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:36 smithi136 ceph-mon[35323]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T02:03:36.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:36 smithi136 ceph-mon[35323]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T02:03:36.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:36 smithi172 ceph-mon[35340]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T02:03:36.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:36 smithi172 ceph-mon[35340]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T02:03:36.333 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:36 smithi103 ceph-mon[30398]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T02:03:36.333 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:36 smithi103 ceph-mon[30398]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T02:03:36.669 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T02:03:37.054 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:37.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:37 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1720800546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T02:03:37.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:37 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1720800546' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T02:03:37.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:37 smithi136 ceph-mon[35323]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T02:03:37.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:37 smithi136 ceph-mon[35323]: pgmap v279: 97 pgs: 16 active+clean, 3 creating+peering, 78 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:37.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:37 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1720800546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T02:03:37.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:37 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1720800546' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T02:03:37.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:37 smithi172 ceph-mon[35340]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T02:03:37.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:37 smithi172 ceph-mon[35340]: pgmap v279: 97 pgs: 16 active+clean, 3 creating+peering, 78 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:37.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:37 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1720800546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T02:03:37.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:37 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1720800546' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T02:03:37.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:37 smithi103 ceph-mon[30398]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T02:03:37.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:37 smithi103 ceph-mon[30398]: pgmap v279: 97 pgs: 16 active+clean, 3 creating+peering, 78 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:37.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.605+0000 7f8ccd800640 1 -- 172.21.15.103:0/2858645455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cc8101de0 msgr2=0x7f8cc81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.605+0000 7f8ccd800640 1 --2- 172.21.15.103:0/2858645455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cc8101de0 0x7f8cc81021c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009900 tx=0x7f8cb802a410 comp rx=0 tx=0).stop 2024-09-16T02:03:37.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.605+0000 7f8ccd800640 1 -- 172.21.15.103:0/2858645455 shutdown_connections 2024-09-16T02:03:37.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.605+0000 7f8ccd800640 1 --2- 172.21.15.103:0/2858645455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cc8106f10 0x7f8cc810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.605+0000 7f8ccd800640 1 --2- 172.21.15.103:0/2858645455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc8102790 0x7f8cc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.605+0000 7f8ccd800640 1 --2- 172.21.15.103:0/2858645455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cc8101de0 0x7f8cc81021c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.605+0000 7f8ccd800640 1 -- 172.21.15.103:0/2858645455 >> 172.21.15.103:0/2858645455 conn(0x7f8cc80fd8c0 msgr2=0x7f8cc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:37.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.605+0000 7f8ccd800640 1 -- 172.21.15.103:0/2858645455 shutdown_connections 2024-09-16T02:03:37.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.605+0000 7f8ccd800640 1 -- 172.21.15.103:0/2858645455 wait complete. 2024-09-16T02:03:37.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.606+0000 7f8ccd800640 1 Processor -- start 2024-09-16T02:03:37.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.606+0000 7f8ccd800640 1 -- start start 2024-09-16T02:03:37.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8ccd800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cc8101de0 0x7f8cc8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8ccd800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cc8102790 0x7f8cc8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8ccd800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc8106f10 0x7f8cc819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8ccd800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8111f80 con 0x7f8cc8102790 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8ccd800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8111e00 con 0x7f8cc8101de0 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8ccd800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8112100 con 0x7f8cc8106f10 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cc8102790 0x7f8cc8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc7400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc8106f10 0x7f8cc819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc7400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc8106f10 0x7f8cc819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:36292/0 (socket says 172.21.15.103:36292) 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cc8102790 0x7f8cc8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45710/0 (socket says 172.21.15.103:45710) 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc7400640 1 -- 172.21.15.103:0/3768799049 learned_addr learned my addr 172.21.15.103:0/3768799049 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc6a00640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cc8101de0 0x7f8cc8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc7400640 1 -- 172.21.15.103:0/3768799049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cc8101de0 msgr2=0x7f8cc8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc7400640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cc8101de0 0x7f8cc8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc7400640 1 -- 172.21.15.103:0/3768799049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cc8102790 msgr2=0x7f8cc8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:37.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc7400640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cc8102790 0x7f8cc8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:37.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.607+0000 7f8cc7400640 1 -- 172.21.15.103:0/3768799049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc819d3e0 con 0x7f8cc8106f10 2024-09-16T02:03:37.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.608+0000 7f8cc7400640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc8106f10 0x7f8cc819cd00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000eb10 tx=0x7f8cc000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:37.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.608+0000 7f8cbf400640 1 -- 172.21.15.103:0/3768799049 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0004b20 con 0x7f8cc8106f10 2024-09-16T02:03:37.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.608+0000 7f8cbf400640 1 -- 172.21.15.103:0/3768799049 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc0004cc0 con 0x7f8cc8106f10 2024-09-16T02:03:37.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.608+0000 7f8cbf400640 1 -- 172.21.15.103:0/3768799049 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0010760 con 0x7f8cc8106f10 2024-09-16T02:03:37.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.608+0000 7f8ccd800640 1 -- 172.21.15.103:0/3768799049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc819d6d0 con 0x7f8cc8106f10 2024-09-16T02:03:37.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.608+0000 7f8ccd800640 1 -- 172.21.15.103:0/3768799049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc81a4fb0 con 0x7f8cc8106f10 2024-09-16T02:03:37.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.609+0000 7f8ccd800640 1 -- 172.21.15.103:0/3768799049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc81021c0 con 0x7f8cc8106f10 2024-09-16T02:03:37.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.611+0000 7f8cbf400640 1 -- 172.21.15.103:0/3768799049 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8cc0005020 con 0x7f8cc8106f10 2024-09-16T02:03:37.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.612+0000 7f8cbf400640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8cac0a3240 0x7f8cac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:37.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.612+0000 7f8cbf400640 1 -- 172.21.15.103:0/3768799049 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7879+0+0 (secure 0 0 0) 0x7f8cc00cfcf0 con 0x7f8cc8106f10 2024-09-16T02:03:37.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.612+0000 7f8cc6a00640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8cac0a3240 0x7f8cac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:37.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.613+0000 7f8cc6a00640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8cac0a3240 0x7f8cac0a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8000c00 tx=0x7f8cb8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:37.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.618+0000 7f8cbf400640 1 -- 172.21.15.103:0/3768799049 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cc009cb20 con 0x7f8cc8106f10 2024-09-16T02:03:37.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:37.828+0000 7f8ccd800640 1 -- 172.21.15.103:0/3768799049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8cc8109b60 con 0x7f8cc8106f10 2024-09-16T02:03:38.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.033+0000 7f8cbf400640 1 -- 172.21.15.103:0/3768799049 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8cc009c420 con 0x7f8cc8106f10 2024-09-16T02:03:38.036 INFO:teuthology.orchestra.run.smithi103.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T02:03:38.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.036+0000 7f8ccd800640 1 -- 172.21.15.103:0/3768799049 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8cac0a3240 msgr2=0x7f8cac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:38.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.036+0000 7f8ccd800640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8cac0a3240 0x7f8cac0a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8000c00 tx=0x7f8cb8005b50 comp rx=0 tx=0).stop 2024-09-16T02:03:38.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.037+0000 7f8ccd800640 1 -- 172.21.15.103:0/3768799049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc8106f10 msgr2=0x7f8cc819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:38.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.037+0000 7f8ccd800640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc8106f10 0x7f8cc819cd00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000eb10 tx=0x7f8cc000c2d0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.037+0000 7f8ccd800640 1 -- 172.21.15.103:0/3768799049 shutdown_connections 2024-09-16T02:03:38.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.037+0000 7f8ccd800640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8cac0a3240 0x7f8cac0a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.037+0000 7f8ccd800640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc8106f10 0x7f8cc819cd00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.037+0000 7f8ccd800640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cc8102790 0x7f8cc8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.037+0000 7f8ccd800640 1 --2- 172.21.15.103:0/3768799049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8cc8101de0 0x7f8cc8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:38.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.037+0000 7f8ccd800640 1 -- 172.21.15.103:0/3768799049 >> 172.21.15.103:0/3768799049 conn(0x7f8cc80fd8c0 msgr2=0x7f8cc80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:38.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.037+0000 7f8ccd800640 1 -- 172.21.15.103:0/3768799049 shutdown_connections 2024-09-16T02:03:38.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:38.037+0000 7f8ccd800640 1 -- 172.21.15.103:0/3768799049 wait complete. 2024-09-16T02:03:38.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:38 smithi136 ceph-mon[35323]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T02:03:38.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:38 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3768799049' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T02:03:38.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:38 smithi136 ceph-mon[35323]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T02:03:38.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:38 smithi136 ceph-mon[35323]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T02:03:38.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:38 smithi172 ceph-mon[35340]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T02:03:38.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:38 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3768799049' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T02:03:38.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:38 smithi172 ceph-mon[35340]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T02:03:38.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:38 smithi172 ceph-mon[35340]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T02:03:38.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:38 smithi103 ceph-mon[30398]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T02:03:38.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:38 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3768799049' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T02:03:38.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:38 smithi103 ceph-mon[30398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T02:03:38.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:38 smithi103 ceph-mon[30398]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T02:03:38.729 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:03:39.100 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:39.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:39 smithi172 ceph-mon[35340]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T02:03:39.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:39 smithi172 ceph-mon[35340]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T02:03:39.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:39 smithi172 ceph-mon[35340]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T02:03:39.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:39 smithi172 ceph-mon[35340]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T02:03:39.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:39 smithi172 ceph-mon[35340]: pgmap v282: 97 pgs: 38 active+clean, 3 creating+peering, 56 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:39.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:39 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] up:active 2024-09-16T02:03:39.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:39 smithi172 ceph-mon[35340]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:03:39.343 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:39 smithi103 ceph-mon[30398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T02:03:39.343 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:39 smithi103 ceph-mon[30398]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T02:03:39.343 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:39 smithi103 ceph-mon[30398]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T02:03:39.344 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:39 smithi103 ceph-mon[30398]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T02:03:39.344 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:39 smithi103 ceph-mon[30398]: pgmap v282: 97 pgs: 38 active+clean, 3 creating+peering, 56 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:39.344 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:39 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] up:active 2024-09-16T02:03:39.344 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:39 smithi103 ceph-mon[30398]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:03:39.446 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:39 smithi136 ceph-mon[35323]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T02:03:39.446 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:39 smithi136 ceph-mon[35323]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T02:03:39.446 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:39 smithi136 ceph-mon[35323]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T02:03:39.446 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:39 smithi136 ceph-mon[35323]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T02:03:39.446 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:39 smithi136 ceph-mon[35323]: pgmap v282: 97 pgs: 38 active+clean, 3 creating+peering, 56 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:03:39.446 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:39 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] up:active 2024-09-16T02:03:39.446 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:39 smithi136 ceph-mon[35323]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:03:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.584+0000 7ffad6800640 1 -- 172.21.15.103:0/1618772027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffad0074620 msgr2=0x7ffad0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.584+0000 7ffad6800640 1 --2- 172.21.15.103:0/1618772027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffad0074620 0x7ffad0074a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac402a480 comp rx=0 tx=0).stop 2024-09-16T02:03:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.584+0000 7ffad6800640 1 -- 172.21.15.103:0/1618772027 shutdown_connections 2024-09-16T02:03:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.584+0000 7ffad6800640 1 --2- 172.21.15.103:0/1618772027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad010e210 0x7ffad0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.584+0000 7ffad6800640 1 --2- 172.21.15.103:0/1618772027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffad0074f40 0x7ffad010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.584+0000 7ffad6800640 1 --2- 172.21.15.103:0/1618772027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffad0074620 0x7ffad0074a00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.584+0000 7ffad6800640 1 -- 172.21.15.103:0/1618772027 >> 172.21.15.103:0/1618772027 conn(0x7ffad00fd710 msgr2=0x7ffad00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.584+0000 7ffad6800640 1 -- 172.21.15.103:0/1618772027 shutdown_connections 2024-09-16T02:03:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.584+0000 7ffad6800640 1 -- 172.21.15.103:0/1618772027 wait complete. 2024-09-16T02:03:39.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.585+0000 7ffad6800640 1 Processor -- start 2024-09-16T02:03:39.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.585+0000 7ffad6800640 1 -- start start 2024-09-16T02:03:39.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffad0074620 0x7ffad019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:39.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffad0074f40 0x7ffad019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:39.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad6800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad010e210 0x7ffad019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:39.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad6800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0114010 con 0x7ffad0074f40 2024-09-16T02:03:39.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad6800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0113e90 con 0x7ffad0074620 2024-09-16T02:03:39.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad6800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0114190 con 0x7ffad010e210 2024-09-16T02:03:39.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad4a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffad0074f40 0x7ffad019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:39.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffad0074620 0x7ffad019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffad0074620 0x7ffad019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:46888/0 (socket says 172.21.15.103:46888) 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad5400640 1 -- 172.21.15.103:0/3768460279 learned_addr learned my addr 172.21.15.103:0/3768460279 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad5e00640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad010e210 0x7ffad019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad4a00640 1 -- 172.21.15.103:0/3768460279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad010e210 msgr2=0x7ffad019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad4a00640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad010e210 0x7ffad019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad4a00640 1 -- 172.21.15.103:0/3768460279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffad0074620 msgr2=0x7ffad019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad4a00640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffad0074620 0x7ffad019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad4a00640 1 -- 172.21.15.103:0/3768460279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad019f460 con 0x7ffad0074f40 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.586+0000 7ffad5400640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffad0074620 0x7ffad019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.587+0000 7ffad4a00640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffad0074f40 0x7ffad019a9f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00e9f0 tx=0x7ffabc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:39.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.587+0000 7fface000640 1 -- 172.21.15.103:0/3768460279 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00cd80 con 0x7ffad0074f40 2024-09-16T02:03:39.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.587+0000 7fface000640 1 -- 172.21.15.103:0/3768460279 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffabc0035f0 con 0x7ffad0074f40 2024-09-16T02:03:39.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.587+0000 7ffad6800640 1 -- 172.21.15.103:0/3768460279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad019f750 con 0x7ffad0074f40 2024-09-16T02:03:39.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.587+0000 7fface000640 1 -- 172.21.15.103:0/3768460279 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc010690 con 0x7ffad0074f40 2024-09-16T02:03:39.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.587+0000 7ffad6800640 1 -- 172.21.15.103:0/3768460279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad01a7030 con 0x7ffad0074f40 2024-09-16T02:03:39.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.588+0000 7ffad6800640 1 -- 172.21.15.103:0/3768460279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad0074a00 con 0x7ffad0074f40 2024-09-16T02:03:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.591+0000 7fface000640 1 -- 172.21.15.103:0/3768460279 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ffabc003b30 con 0x7ffad0074f40 2024-09-16T02:03:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.593+0000 7fface000640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffab80a3240 0x7ffab80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.593+0000 7fface000640 1 -- 172.21.15.103:0/3768460279 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7ffabc0cfb40 con 0x7ffad0074f40 2024-09-16T02:03:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.593+0000 7ffad5400640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffab80a3240 0x7ffab80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.594+0000 7ffad5400640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffab80a3240 0x7ffab80a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffac40096c0 tx=0x7ffac40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:39.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.599+0000 7fface000640 1 -- 172.21.15.103:0/3768460279 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffabc09c910 con 0x7ffad0074f40 2024-09-16T02:03:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.773+0000 7ffad6800640 1 -- 172.21.15.103:0/3768460279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffad002d7c0 con 0x7ffad0074f40 2024-09-16T02:03:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.774+0000 7fface000640 1 -- 172.21.15.103:0/3768460279 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22361 (secure 0 0 0) 0x7ffabc09c210 con 0x7ffad0074f40 2024-09-16T02:03:39.777 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:39.777 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":77,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:03:37.830327+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:03:39.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 -- 172.21.15.103:0/3768460279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffab80a3240 msgr2=0x7ffab80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:39.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffab80a3240 0x7ffab80a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffac40096c0 tx=0x7ffac40046c0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 -- 172.21.15.103:0/3768460279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffad0074f40 msgr2=0x7ffad019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:39.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffad0074f40 0x7ffad019a9f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00e9f0 tx=0x7ffabc00eeb0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 -- 172.21.15.103:0/3768460279 shutdown_connections 2024-09-16T02:03:39.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffab80a3240 0x7ffab80a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad010e210 0x7ffad019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffad0074f40 0x7ffad019a9f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 --2- 172.21.15.103:0/3768460279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffad0074620 0x7ffad019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 -- 172.21.15.103:0/3768460279 >> 172.21.15.103:0/3768460279 conn(0x7ffad00fd710 msgr2=0x7ffad00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 -- 172.21.15.103:0/3768460279 shutdown_connections 2024-09-16T02:03:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:39.777+0000 7ffad6800640 1 -- 172.21.15.103:0/3768460279 wait complete. 2024-09-16T02:03:40.403 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:40 smithi136 ceph-mon[35323]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T02:03:40.403 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:40 smithi136 ceph-mon[35323]: Cluster is now healthy 2024-09-16T02:03:40.404 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:40 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3768460279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:03:40.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:40 smithi103 ceph-mon[30398]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T02:03:40.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:40 smithi103 ceph-mon[30398]: Cluster is now healthy 2024-09-16T02:03:40.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:40 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3768460279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:03:40.545 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:03:40.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:40 smithi172 ceph-mon[35340]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T02:03:40.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:40 smithi172 ceph-mon[35340]: Cluster is now healthy 2024-09-16T02:03:40.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:40 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3768460279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:03:40.773 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:03:40 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-b[66066]: 2024-09-16T02:03:40.623+0000 7fa7a5092640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T02:03:40.916 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:41.376 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:41 smithi103 ceph-mon[30398]: pgmap v283: 97 pgs: 77 active+clean, 20 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-09-16T02:03:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.486+0000 7fc330400640 1 -- 172.21.15.103:0/3032400859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328076720 msgr2=0x7fc328076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.486+0000 7fc330400640 1 --2- 172.21.15.103:0/3032400859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328076720 0x7fc328076b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009900 tx=0x7fc31c02a410 comp rx=0 tx=0).stop 2024-09-16T02:03:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.487+0000 7fc330400640 1 -- 172.21.15.103:0/3032400859 shutdown_connections 2024-09-16T02:03:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.487+0000 7fc330400640 1 --2- 172.21.15.103:0/3032400859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc328112a50 0x7fc328114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.487+0000 7fc330400640 1 --2- 172.21.15.103:0/3032400859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328077040 0x7fc328112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.487+0000 7fc330400640 1 --2- 172.21.15.103:0/3032400859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328076720 0x7fc328076b00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.487+0000 7fc330400640 1 -- 172.21.15.103:0/3032400859 >> 172.21.15.103:0/3032400859 conn(0x7fc3280ffa50 msgr2=0x7fc328101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.487+0000 7fc330400640 1 -- 172.21.15.103:0/3032400859 shutdown_connections 2024-09-16T02:03:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.487+0000 7fc330400640 1 -- 172.21.15.103:0/3032400859 wait complete. 2024-09-16T02:03:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.488+0000 7fc330400640 1 Processor -- start 2024-09-16T02:03:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.488+0000 7fc330400640 1 -- start start 2024-09-16T02:03:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.488+0000 7fc330400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc328076720 0x7fc3281a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.488+0000 7fc330400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328077040 0x7fc3281a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.488+0000 7fc330400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328112a50 0x7fc32819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.488+0000 7fc330400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3281185c0 con 0x7fc328077040 2024-09-16T02:03:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.488+0000 7fc330400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc328118440 con 0x7fc328076720 2024-09-16T02:03:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.488+0000 7fc330400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc328118740 con 0x7fc328112a50 2024-09-16T02:03:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc328076720 0x7fc3281a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328077040 0x7fc3281a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc328076720 0x7fc3281a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:46918/0 (socket says 172.21.15.103:46918) 2024-09-16T02:03:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32da00640 1 -- 172.21.15.103:0/1144903647 learned_addr learned my addr 172.21.15.103:0/1144903647 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32e400640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328112a50 0x7fc32819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32da00640 1 -- 172.21.15.103:0/1144903647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328112a50 msgr2=0x7fc32819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32da00640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328112a50 0x7fc32819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32da00640 1 -- 172.21.15.103:0/1144903647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328077040 msgr2=0x7fc3281a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32da00640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328077040 0x7fc3281a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32da00640 1 -- 172.21.15.103:0/1144903647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32819efe0 con 0x7fc328076720 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.489+0000 7fc32e400640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328112a50 0x7fc32819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.490+0000 7fc32da00640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc328076720 0x7fc3281a46f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc31c006010 tx=0x7fc31c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.490+0000 7fc326000640 1 -- 172.21.15.103:0/1144903647 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c03f030 con 0x7fc328076720 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.490+0000 7fc330400640 1 -- 172.21.15.103:0/1144903647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32819f270 con 0x7fc328076720 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.490+0000 7fc326000640 1 -- 172.21.15.103:0/1144903647 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc31c004440 con 0x7fc328076720 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.490+0000 7fc326000640 1 -- 172.21.15.103:0/1144903647 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c035070 con 0x7fc328076720 2024-09-16T02:03:41.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.490+0000 7fc330400640 1 -- 172.21.15.103:0/1144903647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3281ab5c0 con 0x7fc328076720 2024-09-16T02:03:41.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.491+0000 7fc330400640 1 -- 172.21.15.103:0/1144903647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc328076b00 con 0x7fc328076720 2024-09-16T02:03:41.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.494+0000 7fc326000640 1 -- 172.21.15.103:0/1144903647 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc31c042030 con 0x7fc328076720 2024-09-16T02:03:41.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.496+0000 7fc326000640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc3100a3240 0x7fc3100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:41.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.496+0000 7fc326000640 1 -- 172.21.15.103:0/1144903647 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fc31c0ef880 con 0x7fc328076720 2024-09-16T02:03:41.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.496+0000 7fc32d000640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc3100a3240 0x7fc3100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:41.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.497+0000 7fc32d000640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc3100a3240 0x7fc3100a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc328079220 tx=0x7fc318005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:41.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.500+0000 7fc326000640 1 -- 172.21.15.103:0/1144903647 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc31c0bc6d0 con 0x7fc328076720 2024-09-16T02:03:41.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:41 smithi136 ceph-mon[35323]: pgmap v283: 97 pgs: 77 active+clean, 20 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-09-16T02:03:41.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:41 smithi172 ceph-mon[35340]: pgmap v283: 97 pgs: 77 active+clean, 20 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-09-16T02:03:41.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.683+0000 7fc330400640 1 -- 172.21.15.103:0/1144903647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3281ab7f0 con 0x7fc328076720 2024-09-16T02:03:41.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.685+0000 7fc326000640 1 -- 172.21.15.103:0/1144903647 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22361 (secure 0 0 0) 0x7fc31c0bbfd0 con 0x7fc328076720 2024-09-16T02:03:41.687 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:03:41.688 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":77,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:03:37.830327+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:03:41.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 -- 172.21.15.103:0/1144903647 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc3100a3240 msgr2=0x7fc3100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:41.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc3100a3240 0x7fc3100a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc328079220 tx=0x7fc318005f90 comp rx=0 tx=0).stop 2024-09-16T02:03:41.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 -- 172.21.15.103:0/1144903647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc328076720 msgr2=0x7fc3281a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:41.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc328076720 0x7fc3281a46f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc31c006010 tx=0x7fc31c004040 comp rx=0 tx=0).stop 2024-09-16T02:03:41.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 -- 172.21.15.103:0/1144903647 shutdown_connections 2024-09-16T02:03:41.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc3100a3240 0x7fc3100a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:41.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328112a50 0x7fc32819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:41.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328077040 0x7fc3281a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:41.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 --2- 172.21.15.103:0/1144903647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc328076720 0x7fc3281a46f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:41.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 -- 172.21.15.103:0/1144903647 >> 172.21.15.103:0/1144903647 conn(0x7fc3280ffa50 msgr2=0x7fc328101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:41.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 -- 172.21.15.103:0/1144903647 shutdown_connections 2024-09-16T02:03:41.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:41.687+0000 7fc330400640 1 -- 172.21.15.103:0/1144903647 wait complete. 2024-09-16T02:03:42.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:42 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1144903647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:03:42.404 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-16T02:03:42.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:42 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1144903647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:03:42.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:42 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1144903647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:03:42.792 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.272+0000 7f82eca00640 1 -- 172.21.15.103:0/1215504598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82e81047e0 msgr2=0x7f82e8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.272+0000 7f82eca00640 1 --2- 172.21.15.103:0/1215504598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82e81047e0 0x7f82e8108830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009900 tx=0x7f82d002a410 comp rx=0 tx=0).stop 2024-09-16T02:03:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.272+0000 7f82eca00640 1 -- 172.21.15.103:0/1215504598 shutdown_connections 2024-09-16T02:03:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.272+0000 7f82eca00640 1 --2- 172.21.15.103:0/1215504598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82e8108f60 0x7f82e8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.272+0000 7f82eca00640 1 --2- 172.21.15.103:0/1215504598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82e81047e0 0x7f82e8108830 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.272+0000 7f82eca00640 1 --2- 172.21.15.103:0/1215504598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82e8103e30 0x7f82e8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.272+0000 7f82eca00640 1 -- 172.21.15.103:0/1215504598 >> 172.21.15.103:0/1215504598 conn(0x7f82e80ff8d0 msgr2=0x7f82e8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.272+0000 7f82eca00640 1 -- 172.21.15.103:0/1215504598 shutdown_connections 2024-09-16T02:03:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.272+0000 7f82eca00640 1 -- 172.21.15.103:0/1215504598 wait complete. 2024-09-16T02:03:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82eca00640 1 Processor -- start 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82eca00640 1 -- start start 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82eca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82e8103e30 0x7f82e819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82eca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82e81047e0 0x7f82e819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82eca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82e8108f60 0x7f82e819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82eca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82e8114010 con 0x7f82e8108f60 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82eca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82e8113e90 con 0x7f82e8103e30 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82eca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f82e8114190 con 0x7f82e81047e0 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82e7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82e8108f60 0x7f82e819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82e7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82e8108f60 0x7f82e819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33116/0 (socket says 172.21.15.103:33116) 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.273+0000 7f82e7200640 1 -- 172.21.15.103:0/3486026810 learned_addr learned my addr 172.21.15.103:0/3486026810 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.274+0000 7f82e6800640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82e8103e30 0x7f82e819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.274+0000 7f82dfe00640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82e81047e0 0x7f82e819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.274+0000 7f82e7200640 1 -- 172.21.15.103:0/3486026810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82e81047e0 msgr2=0x7f82e819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.274+0000 7f82e7200640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82e81047e0 0x7f82e819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.274+0000 7f82e7200640 1 -- 172.21.15.103:0/3486026810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82e8103e30 msgr2=0x7f82e819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.274+0000 7f82e7200640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82e8103e30 0x7f82e819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.274+0000 7f82e7200640 1 -- 172.21.15.103:0/3486026810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e819f440 con 0x7f82e8108f60 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.274+0000 7f82e7200640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82e8108f60 0x7f82e819ed60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009fd0 tx=0x7f82e000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.274+0000 7f82df400640 1 -- 172.21.15.103:0/3486026810 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0017070 con 0x7f82e8108f60 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.274+0000 7f82df400640 1 -- 172.21.15.103:0/3486026810 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e0003e50 con 0x7f82e8108f60 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.275+0000 7f82df400640 1 -- 172.21.15.103:0/3486026810 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e00106c0 con 0x7f82e8108f60 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.275+0000 7f82eca00640 1 -- 172.21.15.103:0/3486026810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e819f730 con 0x7f82e8108f60 2024-09-16T02:03:43.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.275+0000 7f82eca00640 1 -- 172.21.15.103:0/3486026810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e81a7010 con 0x7f82e8108f60 2024-09-16T02:03:43.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.276+0000 7f82eca00640 1 -- 172.21.15.103:0/3486026810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e8104210 con 0x7f82e8108f60 2024-09-16T02:03:43.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.277+0000 7f82df400640 1 -- 172.21.15.103:0/3486026810 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f82e0003970 con 0x7f82e8108f60 2024-09-16T02:03:43.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.279+0000 7f82df400640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f82cc0a32c0 0x7f82cc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:43.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.279+0000 7f82df400640 1 -- 172.21.15.103:0/3486026810 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f82e00cf450 con 0x7f82e8108f60 2024-09-16T02:03:43.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.279+0000 7f82e6800640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f82cc0a32c0 0x7f82cc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:43.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.280+0000 7f82e6800640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f82cc0a32c0 0x7f82cc0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82d8005d00 tx=0x7f82d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:43.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.283+0000 7f82df400640 1 -- 172.21.15.103:0/3486026810 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82e009bbd0 con 0x7f82e8108f60 2024-09-16T02:03:43.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:43.494+0000 7f82eca00640 1 -- 172.21.15.103:0/3486026810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f82e819b7f0 con 0x7f82e8108f60 2024-09-16T02:03:43.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:43 smithi103 ceph-mon[30398]: pgmap v284: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-16T02:03:43.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:43 smithi136 ceph-mon[35323]: pgmap v284: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-16T02:03:43.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:43 smithi172 ceph-mon[35340]: pgmap v284: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-16T02:03:44.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.064+0000 7f82df400640 1 -- 172.21.15.103:0/3486026810 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f82e009fed0 con 0x7f82e8108f60 2024-09-16T02:03:44.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.066+0000 7f82eca00640 1 -- 172.21.15.103:0/3486026810 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f82cc0a32c0 msgr2=0x7f82cc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:44.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.066+0000 7f82eca00640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f82cc0a32c0 0x7f82cc0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82d8005d00 tx=0x7f82d800b410 comp rx=0 tx=0).stop 2024-09-16T02:03:44.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.067+0000 7f82eca00640 1 -- 172.21.15.103:0/3486026810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82e8108f60 msgr2=0x7f82e819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:44.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.067+0000 7f82eca00640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82e8108f60 0x7f82e819ed60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009fd0 tx=0x7f82e000b260 comp rx=0 tx=0).stop 2024-09-16T02:03:44.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.067+0000 7f82eca00640 1 -- 172.21.15.103:0/3486026810 shutdown_connections 2024-09-16T02:03:44.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.067+0000 7f82eca00640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f82cc0a32c0 0x7f82cc0a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:44.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.067+0000 7f82eca00640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82e8108f60 0x7f82e819ed60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:44.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.067+0000 7f82eca00640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82e81047e0 0x7f82e819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:44.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.067+0000 7f82eca00640 1 --2- 172.21.15.103:0/3486026810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82e8103e30 0x7f82e819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:44.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.067+0000 7f82eca00640 1 -- 172.21.15.103:0/3486026810 >> 172.21.15.103:0/3486026810 conn(0x7f82e80ff8d0 msgr2=0x7f82e8100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:44.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.067+0000 7f82eca00640 1 -- 172.21.15.103:0/3486026810 shutdown_connections 2024-09-16T02:03:44.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:44.067+0000 7f82eca00640 1 -- 172.21.15.103:0/3486026810 wait complete. 2024-09-16T02:03:44.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:44 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3486026810' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T02:03:44.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:44 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3486026810' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T02:03:44.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:44 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3486026810' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T02:03:44.885 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-16T02:03:44.885 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-16T02:03:45.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:45 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3486026810' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T02:03:45.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:45 smithi103 ceph-mon[30398]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:03:45.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:45 smithi103 ceph-mon[30398]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-16T02:03:45.261 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:45.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:45 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3486026810' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T02:03:45.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:45 smithi136 ceph-mon[35323]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:03:45.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:45 smithi136 ceph-mon[35323]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-16T02:03:45.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:45 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3486026810' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T02:03:45.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:45 smithi172 ceph-mon[35340]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:03:45.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:45 smithi172 ceph-mon[35340]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-16T02:03:45.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.812+0000 7f813f200640 1 -- 172.21.15.103:0/3835915809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f814010e210 msgr2=0x7f8140110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:45.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.812+0000 7f813f200640 1 --2- 172.21.15.103:0/3835915809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f814010e210 0x7f8140110660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8138009ca0 tx=0x7f813802a950 comp rx=0 tx=0).stop 2024-09-16T02:03:45.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.812+0000 7f813f200640 1 -- 172.21.15.103:0/3835915809 shutdown_connections 2024-09-16T02:03:45.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.812+0000 7f813f200640 1 --2- 172.21.15.103:0/3835915809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f814010e210 0x7f8140110660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:45.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.812+0000 7f813f200640 1 --2- 172.21.15.103:0/3835915809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8140074f40 0x7f814010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:45.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.812+0000 7f813f200640 1 --2- 172.21.15.103:0/3835915809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8140074620 0x7f8140074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:45.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.812+0000 7f813f200640 1 -- 172.21.15.103:0/3835915809 >> 172.21.15.103:0/3835915809 conn(0x7f81400fd710 msgr2=0x7f81400ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:45.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.812+0000 7f813f200640 1 -- 172.21.15.103:0/3835915809 shutdown_connections 2024-09-16T02:03:45.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.812+0000 7f813f200640 1 -- 172.21.15.103:0/3835915809 wait complete. 2024-09-16T02:03:45.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.813+0000 7f813f200640 1 Processor -- start 2024-09-16T02:03:45.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.813+0000 7f813f200640 1 -- start start 2024-09-16T02:03:45.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.813+0000 7f813f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8140074620 0x7f814019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:45.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.813+0000 7f813f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8140074f40 0x7f814019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:45.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.813+0000 7f813f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f814010e210 0x7f814019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:45.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.813+0000 7f813f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8140114010 con 0x7f8140074620 2024-09-16T02:03:45.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.813+0000 7f813f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8140113e90 con 0x7f814010e210 2024-09-16T02:03:45.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.813+0000 7f813f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8140114190 con 0x7f8140074f40 2024-09-16T02:03:45.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8140074620 0x7f814019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:45.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f814010e210 0x7f814019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8140074620 0x7f814019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33134/0 (socket says 172.21.15.103:33134) 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813de00640 1 -- 172.21.15.103:0/3710044205 learned_addr learned my addr 172.21.15.103:0/3710044205 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813d400640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8140074f40 0x7f814019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813e800640 1 -- 172.21.15.103:0/3710044205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8140074f40 msgr2=0x7f814019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813e800640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8140074f40 0x7f814019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813e800640 1 -- 172.21.15.103:0/3710044205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8140074620 msgr2=0x7f814019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813e800640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8140074620 0x7f814019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813e800640 1 -- 172.21.15.103:0/3710044205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814019f4e0 con 0x7f814010e210 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813de00640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8140074620 0x7f814019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.814+0000 7f813e800640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f814010e210 0x7f814019ee00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8138005e30 tx=0x7f8138003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.815+0000 7f8136a00640 1 -- 172.21.15.103:0/3710044205 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138034040 con 0x7f814010e210 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.815+0000 7f813f200640 1 -- 172.21.15.103:0/3710044205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814019f770 con 0x7f814010e210 2024-09-16T02:03:45.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.815+0000 7f8136a00640 1 -- 172.21.15.103:0/3710044205 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813802fe20 con 0x7f814010e210 2024-09-16T02:03:45.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.815+0000 7f8136a00640 1 -- 172.21.15.103:0/3710044205 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138004760 con 0x7f814010e210 2024-09-16T02:03:45.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.815+0000 7f813f200640 1 -- 172.21.15.103:0/3710044205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401a70b0 con 0x7f814010e210 2024-09-16T02:03:45.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.816+0000 7f813f200640 1 -- 172.21.15.103:0/3710044205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8140074a00 con 0x7f814010e210 2024-09-16T02:03:45.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.818+0000 7f8136a00640 1 -- 172.21.15.103:0/3710044205 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8138037020 con 0x7f814010e210 2024-09-16T02:03:45.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.819+0000 7f8136a00640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f81240a3240 0x7f81240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:45.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.819+0000 7f8136a00640 1 -- 172.21.15.103:0/3710044205 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f81380f0700 con 0x7f814010e210 2024-09-16T02:03:45.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.819+0000 7f813de00640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f81240a3240 0x7f81240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.820+0000 7f813de00640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f81240a3240 0x7f81240a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81300104e0 tx=0x7f813000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:45.821+0000 7f8136a00640 1 -- 172.21.15.103:0/3710044205 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81380bd4d0 con 0x7f814010e210 2024-09-16T02:03:46.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.034+0000 7f813f200640 1 -- 172.21.15.103:0/3710044205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f8140109390 con 0x7f814010e210 2024-09-16T02:03:46.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.078+0000 7f8136a00640 1 -- 172.21.15.103:0/3710044205 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7f81380bcdd0 con 0x7f814010e210 2024-09-16T02:03:46.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 -- 172.21.15.103:0/3710044205 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f81240a3240 msgr2=0x7f81240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:46.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f81240a3240 0x7f81240a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81300104e0 tx=0x7f813000b410 comp rx=0 tx=0).stop 2024-09-16T02:03:46.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 -- 172.21.15.103:0/3710044205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f814010e210 msgr2=0x7f814019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:46.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f814010e210 0x7f814019ee00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8138005e30 tx=0x7f8138003ce0 comp rx=0 tx=0).stop 2024-09-16T02:03:46.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 -- 172.21.15.103:0/3710044205 shutdown_connections 2024-09-16T02:03:46.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f81240a3240 0x7f81240a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:46.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f814010e210 0x7f814019ee00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:46.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8140074f40 0x7f814019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:46.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 --2- 172.21.15.103:0/3710044205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8140074620 0x7f814019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:46.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 -- 172.21.15.103:0/3710044205 >> 172.21.15.103:0/3710044205 conn(0x7f81400fd710 msgr2=0x7f81400fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:46.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 -- 172.21.15.103:0/3710044205 shutdown_connections 2024-09-16T02:03:46.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:46.081+0000 7f813f200640 1 -- 172.21.15.103:0/3710044205 wait complete. 2024-09-16T02:03:46.402 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:46 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3710044205' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T02:03:46.402 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:46 smithi103 ceph-mon[30398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T02:03:46.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:46 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3710044205' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T02:03:46.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:46 smithi136 ceph-mon[35323]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T02:03:46.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:46 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3710044205' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T02:03:46.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:46 smithi172 ceph-mon[35340]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T02:03:46.889 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-16T02:03:47.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:47 smithi103 ceph-mon[30398]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T02:03:47.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:47 smithi103 ceph-mon[30398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-16T02:03:47.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:47 smithi103 ceph-mon[30398]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T02:03:47.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:47 smithi103 ceph-mon[30398]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:03:47.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:47 smithi103 ceph-mon[30398]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-16T02:03:47.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:47 smithi103 ceph-mon[30398]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-16T02:03:47.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:47 smithi103 ceph-mon[30398]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-16T02:03:47.266 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:47.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:47 smithi136 ceph-mon[35323]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T02:03:47.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:47 smithi136 ceph-mon[35323]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-16T02:03:47.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:47 smithi136 ceph-mon[35323]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T02:03:47.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:47 smithi136 ceph-mon[35323]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:03:47.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:47 smithi136 ceph-mon[35323]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-16T02:03:47.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:47 smithi136 ceph-mon[35323]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-16T02:03:47.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:47 smithi136 ceph-mon[35323]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-16T02:03:47.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:47 smithi172 ceph-mon[35340]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T02:03:47.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:47 smithi172 ceph-mon[35340]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-16T02:03:47.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:47 smithi172 ceph-mon[35340]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T02:03:47.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:47 smithi172 ceph-mon[35340]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T02:03:47.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:47 smithi172 ceph-mon[35340]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-16T02:03:47.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:47 smithi172 ceph-mon[35340]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-16T02:03:47.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:47 smithi172 ceph-mon[35340]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-16T02:03:47.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.727+0000 7f23a5400640 1 -- 172.21.15.103:0/3814982982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a0102790 msgr2=0x7f23a01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:47.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.727+0000 7f23a5400640 1 --2- 172.21.15.103:0/3814982982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a0102790 0x7f23a01067e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2388009900 tx=0x7f238802a410 comp rx=0 tx=0).stop 2024-09-16T02:03:47.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.728+0000 7f23a5400640 1 -- 172.21.15.103:0/3814982982 shutdown_connections 2024-09-16T02:03:47.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.728+0000 7f23a5400640 1 --2- 172.21.15.103:0/3814982982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0106f10 0x7f23a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:47.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.728+0000 7f23a5400640 1 --2- 172.21.15.103:0/3814982982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a0102790 0x7f23a01067e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:47.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.728+0000 7f23a5400640 1 --2- 172.21.15.103:0/3814982982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a0101de0 0x7f23a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:47.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.728+0000 7f23a5400640 1 -- 172.21.15.103:0/3814982982 >> 172.21.15.103:0/3814982982 conn(0x7f23a00fd8c0 msgr2=0x7f23a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:47.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.728+0000 7f23a5400640 1 -- 172.21.15.103:0/3814982982 shutdown_connections 2024-09-16T02:03:47.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.728+0000 7f23a5400640 1 -- 172.21.15.103:0/3814982982 wait complete. 2024-09-16T02:03:47.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.729+0000 7f23a5400640 1 Processor -- start 2024-09-16T02:03:47.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.729+0000 7f23a5400640 1 -- start start 2024-09-16T02:03:47.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f23a5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a0101de0 0x7f23a019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:47.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f23a5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0102790 0x7f23a019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f23a5400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a0106f10 0x7f23a019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f23a5400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23a0111fc0 con 0x7f23a0101de0 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f23a5400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f23a0111e40 con 0x7f23a0102790 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f23a5400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23a0112140 con 0x7f23a0106f10 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f239e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a0101de0 0x7f23a019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f239e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a0101de0 0x7f23a019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33146/0 (socket says 172.21.15.103:33146) 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f239e800640 1 -- 172.21.15.103:0/2181548495 learned_addr learned my addr 172.21.15.103:0/2181548495 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f2395e00640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0102790 0x7f23a019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f239f200640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a0106f10 0x7f23a019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f239e800640 1 -- 172.21.15.103:0/2181548495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a0106f10 msgr2=0x7f23a019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f239e800640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a0106f10 0x7f23a019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:47.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f239e800640 1 -- 172.21.15.103:0/2181548495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0102790 msgr2=0x7f23a019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:47.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f239e800640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0102790 0x7f23a019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:47.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.730+0000 7f239e800640 1 -- 172.21.15.103:0/2181548495 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a019f6f0 con 0x7f23a0101de0 2024-09-16T02:03:47.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.731+0000 7f239e800640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a0101de0 0x7f23a019a740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f239000eb90 tx=0x7f239000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:47.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.731+0000 7f2397e00640 1 -- 172.21.15.103:0/2181548495 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390004d20 con 0x7f23a0101de0 2024-09-16T02:03:47.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.731+0000 7f2397e00640 1 -- 172.21.15.103:0/2181548495 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f239000ee90 con 0x7f23a0101de0 2024-09-16T02:03:47.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.731+0000 7f23a5400640 1 -- 172.21.15.103:0/2181548495 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a019f9e0 con 0x7f23a0101de0 2024-09-16T02:03:47.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.731+0000 7f2397e00640 1 -- 172.21.15.103:0/2181548495 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390010830 con 0x7f23a0101de0 2024-09-16T02:03:47.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.731+0000 7f23a5400640 1 -- 172.21.15.103:0/2181548495 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a01a72c0 con 0x7f23a0101de0 2024-09-16T02:03:47.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.732+0000 7f23a5400640 1 -- 172.21.15.103:0/2181548495 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a01021c0 con 0x7f23a0101de0 2024-09-16T02:03:47.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.734+0000 7f2397e00640 1 -- 172.21.15.103:0/2181548495 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2390005020 con 0x7f23a0101de0 2024-09-16T02:03:47.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.736+0000 7f2397e00640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23840a32c0 0x7f23840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:47.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.736+0000 7f2397e00640 1 -- 172.21.15.103:0/2181548495 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f23900cfe00 con 0x7f23a0101de0 2024-09-16T02:03:47.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.736+0000 7f2395e00640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23840a32c0 0x7f23840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:47.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.737+0000 7f2395e00640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23840a32c0 0x7f23840a5780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2388000c00 tx=0x7f2388005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:47.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.740+0000 7f2397e00640 1 -- 172.21.15.103:0/2181548495 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f239009cbd0 con 0x7f23a0101de0 2024-09-16T02:03:47.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:47.951+0000 7f23a5400640 1 -- 172.21.15.103:0/2181548495 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f23a01a7490 con 0x7f23a0101de0 2024-09-16T02:03:48.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.090+0000 7f2397e00640 1 -- 172.21.15.103:0/2181548495 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f239009c4d0 con 0x7f23a0101de0 2024-09-16T02:03:48.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 -- 172.21.15.103:0/2181548495 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23840a32c0 msgr2=0x7f23840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:48.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23840a32c0 0x7f23840a5780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2388000c00 tx=0x7f2388005b50 comp rx=0 tx=0).stop 2024-09-16T02:03:48.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 -- 172.21.15.103:0/2181548495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a0101de0 msgr2=0x7f23a019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:48.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a0101de0 0x7f23a019a740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f239000eb90 tx=0x7f239000c6a0 comp rx=0 tx=0).stop 2024-09-16T02:03:48.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 -- 172.21.15.103:0/2181548495 shutdown_connections 2024-09-16T02:03:48.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f23840a32c0 0x7f23840a5780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:48.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a0106f10 0x7f23a019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:48.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0102790 0x7f23a019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:48.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 --2- 172.21.15.103:0/2181548495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a0101de0 0x7f23a019a740 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:48.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 -- 172.21.15.103:0/2181548495 >> 172.21.15.103:0/2181548495 conn(0x7f23a00fd8c0 msgr2=0x7f23a00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:48.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 -- 172.21.15.103:0/2181548495 shutdown_connections 2024-09-16T02:03:48.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:48.092+0000 7f23a5400640 1 -- 172.21.15.103:0/2181548495 wait complete. 2024-09-16T02:03:48.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:48 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] up:active 2024-09-16T02:03:48.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:48 smithi103 ceph-mon[30398]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T02:03:48.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:48 smithi103 ceph-mon[30398]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T02:03:48.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:48 smithi103 ceph-mon[30398]: Cluster is now healthy 2024-09-16T02:03:48.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:48 smithi103 ceph-mon[30398]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-16T02:03:48.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:48 smithi103 ceph-mon[30398]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-16T02:03:48.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:48 smithi103 ceph-mon[30398]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-16T02:03:48.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:48 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2181548495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T02:03:48.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:48 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] up:active 2024-09-16T02:03:48.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:48 smithi172 ceph-mon[35340]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T02:03:48.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:48 smithi172 ceph-mon[35340]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T02:03:48.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:48 smithi172 ceph-mon[35340]: Cluster is now healthy 2024-09-16T02:03:48.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:48 smithi172 ceph-mon[35340]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-16T02:03:48.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:48 smithi172 ceph-mon[35340]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-16T02:03:48.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:48 smithi172 ceph-mon[35340]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-16T02:03:48.296 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:48 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2181548495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T02:03:48.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:48 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] up:active 2024-09-16T02:03:48.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:48 smithi136 ceph-mon[35323]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T02:03:48.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:48 smithi136 ceph-mon[35323]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T02:03:48.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:48 smithi136 ceph-mon[35323]: Cluster is now healthy 2024-09-16T02:03:48.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:48 smithi136 ceph-mon[35323]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-16T02:03:48.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:48 smithi136 ceph-mon[35323]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-16T02:03:48.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:48 smithi136 ceph-mon[35323]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-16T02:03:48.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:48 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2181548495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T02:03:48.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-16T02:03:49.146 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:49.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:49 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2181548495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T02:03:49.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:49 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] up:active 2024-09-16T02:03:49.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:49 smithi103 ceph-mon[30398]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-16T02:03:49.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:49 smithi103 ceph-mon[30398]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T02:03:49.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:49 smithi103 ceph-mon[30398]: pgmap v287: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-09-16T02:03:49.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:49 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2181548495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T02:03:49.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:49 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] up:active 2024-09-16T02:03:49.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:49 smithi136 ceph-mon[35323]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-16T02:03:49.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:49 smithi136 ceph-mon[35323]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T02:03:49.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:49 smithi136 ceph-mon[35323]: pgmap v287: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-09-16T02:03:49.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:49 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2181548495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T02:03:49.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:49 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] up:active 2024-09-16T02:03:49.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:49 smithi172 ceph-mon[35340]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-16T02:03:49.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:49 smithi172 ceph-mon[35340]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T02:03:49.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:49 smithi172 ceph-mon[35340]: pgmap v287: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-09-16T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.586+0000 7f074b800640 1 -- 172.21.15.103:0/138738732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0744102790 msgr2=0x7f07441067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.586+0000 7f074b800640 1 --2- 172.21.15.103:0/138738732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0744102790 0x7f07441067e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f072c009900 tx=0x7f072c02a410 comp rx=0 tx=0).stop 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.587+0000 7f074b800640 1 -- 172.21.15.103:0/138738732 shutdown_connections 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.587+0000 7f074b800640 1 --2- 172.21.15.103:0/138738732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0744106f10 0x7f074410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.587+0000 7f074b800640 1 --2- 172.21.15.103:0/138738732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0744102790 0x7f07441067e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.587+0000 7f074b800640 1 --2- 172.21.15.103:0/138738732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0744101de0 0x7f07441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.587+0000 7f074b800640 1 -- 172.21.15.103:0/138738732 >> 172.21.15.103:0/138738732 conn(0x7f07440fd8c0 msgr2=0x7f07440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.587+0000 7f074b800640 1 -- 172.21.15.103:0/138738732 shutdown_connections 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.587+0000 7f074b800640 1 -- 172.21.15.103:0/138738732 wait complete. 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.588+0000 7f074b800640 1 Processor -- start 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.588+0000 7f074b800640 1 -- start start 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.588+0000 7f074b800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0744101de0 0x7f07440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.588+0000 7f074b800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0744102790 0x7f0744074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.588+0000 7f074b800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0744106f10 0x7f0744074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.588+0000 7f074b800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0744111fc0 con 0x7f0744101de0 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.588+0000 7f074b800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0744111e40 con 0x7f0744106f10 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.588+0000 7f074b800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0744112140 con 0x7f0744102790 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.588+0000 7f0748e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0744101de0 0x7f07440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.588+0000 7f0748e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0744101de0 0x7f07440744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49286/0 (socket says 172.21.15.103:49286) 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0748e00640 1 -- 172.21.15.103:0/3779911455 learned_addr learned my addr 172.21.15.103:0/3779911455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f073be00640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0744102790 0x7f0744074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0749800640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0744106f10 0x7f0744074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0748e00640 1 -- 172.21.15.103:0/3779911455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0744102790 msgr2=0x7f0744074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0748e00640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0744102790 0x7f0744074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:49.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0748e00640 1 -- 172.21.15.103:0/3779911455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0744106f10 msgr2=0x7f0744074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0748e00640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0744106f10 0x7f0744074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0748e00640 1 -- 172.21.15.103:0/3779911455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07441aaed0 con 0x7f0744101de0 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0749800640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0744106f10 0x7f0744074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0748e00640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0744101de0 0x7f07440744f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f073400d770 tx=0x7f073400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0742000640 1 -- 172.21.15.103:0/3779911455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734017070 con 0x7f0744101de0 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f074b800640 1 -- 172.21.15.103:0/3779911455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07441ab100 con 0x7f0744101de0 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0742000640 1 -- 172.21.15.103:0/3779911455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0734005500 con 0x7f0744101de0 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.589+0000 7f0742000640 1 -- 172.21.15.103:0/3779911455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734004e00 con 0x7f0744101de0 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.590+0000 7f074b800640 1 -- 172.21.15.103:0/3779911455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07441ab610 con 0x7f0744101de0 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.591+0000 7f074b800640 1 -- 172.21.15.103:0/3779911455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07441021c0 con 0x7f0744101de0 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.593+0000 7f0742000640 1 -- 172.21.15.103:0/3779911455 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0734004fa0 con 0x7f0744101de0 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.594+0000 7f0742000640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f07280a32c0 0x7f07280a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:49.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.594+0000 7f0742000640 1 -- 172.21.15.103:0/3779911455 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f07340cf220 con 0x7f0744101de0 2024-09-16T02:03:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.594+0000 7f073be00640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f07280a32c0 0x7f07280a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.595+0000 7f073be00640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f07280a32c0 0x7f07280a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f072c000c00 tx=0x7f072c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.599+0000 7f0742000640 1 -- 172.21.15.103:0/3779911455 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f073409c090 con 0x7f0744101de0 2024-09-16T02:03:49.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:49.810+0000 7f074b800640 1 -- 172.21.15.103:0/3779911455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f074410a0f0 con 0x7f0744101de0 2024-09-16T02:03:50.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.102+0000 7f0742000640 1 -- 172.21.15.103:0/3779911455 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f073409b990 con 0x7f0744101de0 2024-09-16T02:03:50.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.104+0000 7f074b800640 1 -- 172.21.15.103:0/3779911455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f07280a32c0 msgr2=0x7f07280a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:50.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.104+0000 7f074b800640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f07280a32c0 0x7f07280a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f072c000c00 tx=0x7f072c005a20 comp rx=0 tx=0).stop 2024-09-16T02:03:50.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.104+0000 7f074b800640 1 -- 172.21.15.103:0/3779911455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0744101de0 msgr2=0x7f07440744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:50.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.104+0000 7f074b800640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0744101de0 0x7f07440744f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f073400d770 tx=0x7f073400dc30 comp rx=0 tx=0).stop 2024-09-16T02:03:50.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.105+0000 7f074b800640 1 -- 172.21.15.103:0/3779911455 shutdown_connections 2024-09-16T02:03:50.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.105+0000 7f074b800640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f07280a32c0 0x7f07280a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:50.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.105+0000 7f074b800640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0744106f10 0x7f0744074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:50.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.105+0000 7f074b800640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0744102790 0x7f0744074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:50.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.105+0000 7f074b800640 1 --2- 172.21.15.103:0/3779911455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0744101de0 0x7f07440744f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:50.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.105+0000 7f074b800640 1 -- 172.21.15.103:0/3779911455 >> 172.21.15.103:0/3779911455 conn(0x7f07440fd8c0 msgr2=0x7f07440fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:50.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.105+0000 7f074b800640 1 -- 172.21.15.103:0/3779911455 shutdown_connections 2024-09-16T02:03:50.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:50.105+0000 7f074b800640 1 -- 172.21.15.103:0/3779911455 wait complete. 2024-09-16T02:03:50.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:50 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3779911455' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T02:03:50.438 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:50 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3779911455' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T02:03:50.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:50 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3779911455' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T02:03:50.910 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-16T02:03:50.910 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-16T02:03:51.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:51 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3779911455' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T02:03:51.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:51 smithi103 ceph-mon[30398]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T02:03:51.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:51 smithi103 ceph-mon[30398]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-16T02:03:51.284 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:51.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:51 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3779911455' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T02:03:51.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:51 smithi136 ceph-mon[35323]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T02:03:51.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:51 smithi136 ceph-mon[35323]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-16T02:03:51.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:51 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3779911455' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T02:03:51.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:51 smithi172 ceph-mon[35340]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T02:03:51.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:51 smithi172 ceph-mon[35340]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-16T02:03:51.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.812+0000 7f0678200640 1 -- 172.21.15.103:0/1704008253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0670106f10 msgr2=0x7f067010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:51.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.812+0000 7f0678200640 1 --2- 172.21.15.103:0/1704008253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0670106f10 0x7f067010ea90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0668009be0 tx=0x7f066802a6c0 comp rx=0 tx=0).stop 2024-09-16T02:03:51.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.813+0000 7f0678200640 1 -- 172.21.15.103:0/1704008253 shutdown_connections 2024-09-16T02:03:51.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.813+0000 7f0678200640 1 --2- 172.21.15.103:0/1704008253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0670106f10 0x7f067010ea90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:51.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.813+0000 7f0678200640 1 --2- 172.21.15.103:0/1704008253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0670102790 0x7f06701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:51.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.813+0000 7f0678200640 1 --2- 172.21.15.103:0/1704008253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0670101de0 0x7f06701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:51.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.813+0000 7f0678200640 1 -- 172.21.15.103:0/1704008253 >> 172.21.15.103:0/1704008253 conn(0x7f06700fd8c0 msgr2=0x7f06700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:51.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.813+0000 7f0678200640 1 -- 172.21.15.103:0/1704008253 shutdown_connections 2024-09-16T02:03:51.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.813+0000 7f0678200640 1 -- 172.21.15.103:0/1704008253 wait complete. 2024-09-16T02:03:51.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0678200640 1 Processor -- start 2024-09-16T02:03:51.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0678200640 1 -- start start 2024-09-16T02:03:51.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0678200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0670101de0 0x7f06701a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:51.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0678200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0670102790 0x7f06701a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:51.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0678200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0670106f10 0x7f067010dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:51.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0678200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0670111f20 con 0x7f0670106f10 2024-09-16T02:03:51.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0678200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0670111da0 con 0x7f0670102790 2024-09-16T02:03:51.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0678200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f06701120a0 con 0x7f0670101de0 2024-09-16T02:03:51.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0676200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0670106f10 0x7f067010dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:51.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0676200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0670106f10 0x7f067010dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49304/0 (socket says 172.21.15.103:49304) 2024-09-16T02:03:51.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.814+0000 7f0676200640 1 -- 172.21.15.103:0/3740537364 learned_addr learned my addr 172.21.15.103:0/3740537364 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:51.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.815+0000 7f0674e00640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0670102790 0x7f06701a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:51.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.815+0000 7f0675800640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0670101de0 0x7f06701a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:51.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.815+0000 7f0676200640 1 -- 172.21.15.103:0/3740537364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0670101de0 msgr2=0x7f06701a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:51.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.815+0000 7f0676200640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0670101de0 0x7f06701a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:51.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.815+0000 7f0676200640 1 -- 172.21.15.103:0/3740537364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0670102790 msgr2=0x7f06701a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:51.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.815+0000 7f0676200640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0670102790 0x7f06701a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:51.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.815+0000 7f0676200640 1 -- 172.21.15.103:0/3740537364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067010e380 con 0x7f0670106f10 2024-09-16T02:03:51.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.815+0000 7f0676200640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0670106f10 0x7f067010dc30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0668034040 tx=0x7f06680305b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:51.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.815+0000 7f066e000640 1 -- 172.21.15.103:0/3740537364 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668041020 con 0x7f0670106f10 2024-09-16T02:03:51.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.816+0000 7f066e000640 1 -- 172.21.15.103:0/3740537364 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0668037040 con 0x7f0670106f10 2024-09-16T02:03:51.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.816+0000 7f0678200640 1 -- 172.21.15.103:0/3740537364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067010e610 con 0x7f0670106f10 2024-09-16T02:03:51.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.816+0000 7f066e000640 1 -- 172.21.15.103:0/3740537364 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668036070 con 0x7f0670106f10 2024-09-16T02:03:51.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.816+0000 7f0678200640 1 -- 172.21.15.103:0/3740537364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06701ab5a0 con 0x7f0670106f10 2024-09-16T02:03:51.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.817+0000 7f0678200640 1 -- 172.21.15.103:0/3740537364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06701021c0 con 0x7f0670106f10 2024-09-16T02:03:51.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.819+0000 7f066e000640 1 -- 172.21.15.103:0/3740537364 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0668044020 con 0x7f0670106f10 2024-09-16T02:03:51.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.820+0000 7f066e000640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06580a32c0 0x7f06580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:51.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.820+0000 7f066e000640 1 -- 172.21.15.103:0/3740537364 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f06680f0200 con 0x7f0670106f10 2024-09-16T02:03:51.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.820+0000 7f0675800640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06580a32c0 0x7f06580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:51.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.821+0000 7f0675800640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06580a32c0 0x7f06580a5780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0664009bd0 tx=0x7f0664005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:51.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:51.824+0000 7f066e000640 1 -- 172.21.15.103:0/3740537364 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f066804b020 con 0x7f0670106f10 2024-09-16T02:03:52.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.022+0000 7f0678200640 1 -- 172.21.15.103:0/3740537364 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0670062630 con 0x7f06580a32c0 2024-09-16T02:03:52.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.053+0000 7f066e000640 1 -- 172.21.15.103:0/3740537364 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0670062630 con 0x7f06580a32c0 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.055+0000 7f0678200640 1 -- 172.21.15.103:0/3740537364 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06580a32c0 msgr2=0x7f06580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.055+0000 7f0678200640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06580a32c0 0x7f06580a5780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0664009bd0 tx=0x7f0664005f90 comp rx=0 tx=0).stop 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.055+0000 7f0678200640 1 -- 172.21.15.103:0/3740537364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0670106f10 msgr2=0x7f067010dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.055+0000 7f0678200640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0670106f10 0x7f067010dc30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0668034040 tx=0x7f06680305b0 comp rx=0 tx=0).stop 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.055+0000 7f0678200640 1 -- 172.21.15.103:0/3740537364 shutdown_connections 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.055+0000 7f0678200640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06580a32c0 0x7f06580a5780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.055+0000 7f0678200640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0670106f10 0x7f067010dc30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.055+0000 7f0678200640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0670102790 0x7f06701a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.055+0000 7f0678200640 1 --2- 172.21.15.103:0/3740537364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0670101de0 0x7f06701a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.055+0000 7f0678200640 1 -- 172.21.15.103:0/3740537364 >> 172.21.15.103:0/3740537364 conn(0x7f06700fd8c0 msgr2=0x7f06700feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.056+0000 7f0678200640 1 -- 172.21.15.103:0/3740537364 shutdown_connections 2024-09-16T02:03:52.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:52.056+0000 7f0678200640 1 -- 172.21.15.103:0/3740537364 wait complete. 2024-09-16T02:03:52.377 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:03:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:03:52.029+0000 7fce31200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:03:52.378 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:03:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:03:52.029+0000 7fce31200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:03:52.378 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:03:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:03:52.029+0000 7fce31200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:03:52.378 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:03:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:03:52.029+0000 7fce31200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:03:52.378 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:03:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:03:52.029+0000 7fce31200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:03:52.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-16T02:03:53.006 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:53.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.573+0000 7fe0fa000640 1 -- 172.21.15.103:0/3107437885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f4109120 msgr2=0x7fe0f4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:53.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.573+0000 7fe0fa000640 1 --2- 172.21.15.103:0/3107437885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f4109120 0x7fe0f4110ca0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009900 tx=0x7fe0ec02a410 comp rx=0 tx=0).stop 2024-09-16T02:03:53.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.573+0000 7fe0fa000640 1 -- 172.21.15.103:0/3107437885 shutdown_connections 2024-09-16T02:03:53.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.573+0000 7fe0fa000640 1 --2- 172.21.15.103:0/3107437885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f4109120 0x7fe0f4110ca0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:53.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.573+0000 7fe0fa000640 1 --2- 172.21.15.103:0/3107437885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0f41049a0 0x7fe0f41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:53.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.573+0000 7fe0fa000640 1 --2- 172.21.15.103:0/3107437885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f4103ff0 0x7fe0f41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:53.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.573+0000 7fe0fa000640 1 -- 172.21.15.103:0/3107437885 >> 172.21.15.103:0/3107437885 conn(0x7fe0f40ffa90 msgr2=0x7fe0f4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:53.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.573+0000 7fe0fa000640 1 -- 172.21.15.103:0/3107437885 shutdown_connections 2024-09-16T02:03:53.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.573+0000 7fe0fa000640 1 -- 172.21.15.103:0/3107437885 wait complete. 2024-09-16T02:03:53.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.574+0000 7fe0fa000640 1 Processor -- start 2024-09-16T02:03:53.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.574+0000 7fe0fa000640 1 -- start start 2024-09-16T02:03:53.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0fa000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f4103ff0 0x7fe0f419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:53.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0fa000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0f41049a0 0x7fe0f419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0fa000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f4109120 0x7fe0f419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0fa000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f4114130 con 0x7fe0f4103ff0 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0fa000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f4113fb0 con 0x7fe0f4109120 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0fa000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f41142b0 con 0x7fe0f41049a0 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0f3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f4103ff0 0x7fe0f419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0f3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f4103ff0 0x7fe0f419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49324/0 (socket says 172.21.15.103:49324) 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0f3400640 1 -- 172.21.15.103:0/1541899922 learned_addr learned my addr 172.21.15.103:0/1541899922 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0f3e00640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f4109120 0x7fe0f419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0eaa00640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0f41049a0 0x7fe0f419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0f3400640 1 -- 172.21.15.103:0/1541899922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0f41049a0 msgr2=0x7fe0f419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0f3400640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0f41049a0 0x7fe0f419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0f3400640 1 -- 172.21.15.103:0/1541899922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f4109120 msgr2=0x7fe0f419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:53.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0f3400640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f4109120 0x7fe0f419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:53.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0f3400640 1 -- 172.21.15.103:0/1541899922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f419f540 con 0x7fe0f4103ff0 2024-09-16T02:03:53.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.575+0000 7fe0eaa00640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0f41049a0 0x7fe0f419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:03:53.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.576+0000 7fe0f3400640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f4103ff0 0x7fe0f419a590 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400ea60 tx=0x7fe0e400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:53.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.576+0000 7fe0f0c00640 1 -- 172.21.15.103:0/1541899922 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e400cde0 con 0x7fe0f4103ff0 2024-09-16T02:03:53.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.576+0000 7fe0f0c00640 1 -- 172.21.15.103:0/1541899922 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e40035f0 con 0x7fe0f4103ff0 2024-09-16T02:03:53.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.576+0000 7fe0fa000640 1 -- 172.21.15.103:0/1541899922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f419f830 con 0x7fe0f4103ff0 2024-09-16T02:03:53.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.576+0000 7fe0f0c00640 1 -- 172.21.15.103:0/1541899922 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4010670 con 0x7fe0f4103ff0 2024-09-16T02:03:53.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.576+0000 7fe0fa000640 1 -- 172.21.15.103:0/1541899922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f41a7110 con 0x7fe0f4103ff0 2024-09-16T02:03:53.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.577+0000 7fe0fa000640 1 -- 172.21.15.103:0/1541899922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f41043d0 con 0x7fe0f4103ff0 2024-09-16T02:03:53.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.579+0000 7fe0f0c00640 1 -- 172.21.15.103:0/1541899922 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe0e40037a0 con 0x7fe0f4103ff0 2024-09-16T02:03:53.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.580+0000 7fe0f0c00640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0d80a3350 0x7fe0d80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:53.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.580+0000 7fe0f0c00640 1 -- 172.21.15.103:0/1541899922 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fe0e40cfd10 con 0x7fe0f4103ff0 2024-09-16T02:03:53.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.581+0000 7fe0eaa00640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0d80a3350 0x7fe0d80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:53.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.581+0000 7fe0eaa00640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0d80a3350 0x7fe0d80a5810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe0f410aa90 tx=0x7fe0dc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:53.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.585+0000 7fe0f0c00640 1 -- 172.21.15.103:0/1541899922 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0e40d5020 con 0x7fe0f4103ff0 2024-09-16T02:03:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:53 smithi103 ceph-mon[30398]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:53.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:53 smithi103 ceph-mon[30398]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-16T02:03:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:53 smithi136 ceph-mon[35323]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:53.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:53 smithi136 ceph-mon[35323]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-16T02:03:53.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.785+0000 7fe0fa000640 1 -- 172.21.15.103:0/1541899922 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe0f4100d60 con 0x7fe0d80a3350 2024-09-16T02:03:53.794 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:53 smithi172 ceph-mon[35340]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:53 smithi172 ceph-mon[35340]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-16T02:03:53.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.813+0000 7fe0f0c00640 1 -- 172.21.15.103:0/1541899922 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe0f4100d60 con 0x7fe0d80a3350 2024-09-16T02:03:53.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.815+0000 7fe0fa000640 1 -- 172.21.15.103:0/1541899922 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0d80a3350 msgr2=0x7fe0d80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:53.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.815+0000 7fe0fa000640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0d80a3350 0x7fe0d80a5810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe0f410aa90 tx=0x7fe0dc009500 comp rx=0 tx=0).stop 2024-09-16T02:03:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.815+0000 7fe0fa000640 1 -- 172.21.15.103:0/1541899922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f4103ff0 msgr2=0x7fe0f419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.815+0000 7fe0fa000640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f4103ff0 0x7fe0f419a590 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400ea60 tx=0x7fe0e400ef20 comp rx=0 tx=0).stop 2024-09-16T02:03:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.815+0000 7fe0fa000640 1 -- 172.21.15.103:0/1541899922 shutdown_connections 2024-09-16T02:03:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.815+0000 7fe0fa000640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0d80a3350 0x7fe0d80a5810 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.815+0000 7fe0fa000640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0f4109120 0x7fe0f419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.815+0000 7fe0fa000640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0f41049a0 0x7fe0f419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.815+0000 7fe0fa000640 1 --2- 172.21.15.103:0/1541899922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f4103ff0 0x7fe0f419a590 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.815+0000 7fe0fa000640 1 -- 172.21.15.103:0/1541899922 >> 172.21.15.103:0/1541899922 conn(0x7fe0f40ffa90 msgr2=0x7fe0f4101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.816+0000 7fe0fa000640 1 -- 172.21.15.103:0/1541899922 shutdown_connections 2024-09-16T02:03:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:53.816+0000 7fe0fa000640 1 -- 172.21.15.103:0/1541899922 wait complete. 2024-09-16T02:03:54.484 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-16T02:03:54.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:54 smithi103 ceph-mon[30398]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T02:03:54.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:54 smithi172 ceph-mon[35340]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T02:03:54.857 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:55.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:54 smithi136 ceph-mon[35323]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T02:03:55.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.379+0000 7f21c5200640 1 -- 172.21.15.103:0/1897047704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c0074f40 msgr2=0x7f21c010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:55.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.379+0000 7f21c5200640 1 --2- 172.21.15.103:0/1897047704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c0074f40 0x7f21c010dcd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009900 tx=0x7f21ac02a480 comp rx=0 tx=0).stop 2024-09-16T02:03:55.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.379+0000 7f21c5200640 1 -- 172.21.15.103:0/1897047704 shutdown_connections 2024-09-16T02:03:55.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.379+0000 7f21c5200640 1 --2- 172.21.15.103:0/1897047704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21c010e210 0x7f21c0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:55.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.379+0000 7f21c5200640 1 --2- 172.21.15.103:0/1897047704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c0074f40 0x7f21c010dcd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:55.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.379+0000 7f21c5200640 1 --2- 172.21.15.103:0/1897047704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21c0074620 0x7f21c0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:55.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.379+0000 7f21c5200640 1 -- 172.21.15.103:0/1897047704 >> 172.21.15.103:0/1897047704 conn(0x7f21c00fd710 msgr2=0x7f21c00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:55.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.379+0000 7f21c5200640 1 -- 172.21.15.103:0/1897047704 shutdown_connections 2024-09-16T02:03:55.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.379+0000 7f21c5200640 1 -- 172.21.15.103:0/1897047704 wait complete. 2024-09-16T02:03:55.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.380+0000 7f21c5200640 1 Processor -- start 2024-09-16T02:03:55.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.380+0000 7f21c5200640 1 -- start start 2024-09-16T02:03:55.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.380+0000 7f21c5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c0074620 0x7f21c019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:55.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21c5200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21c0074f40 0x7f21c019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:55.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21c5200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21c010e210 0x7f21c019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:55.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21c5200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01140d0 con 0x7f21c0074620 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21c5200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0113f50 con 0x7f21c010e210 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21c5200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0114250 con 0x7f21c0074f40 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21bf400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c0074620 0x7f21c019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21bf400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c0074620 0x7f21c019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49346/0 (socket says 172.21.15.103:49346) 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21bf400640 1 -- 172.21.15.103:0/2242434213 learned_addr learned my addr 172.21.15.103:0/2242434213 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21bea00640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21c0074f40 0x7f21c019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21bfe00640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21c010e210 0x7f21c019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21bf400640 1 -- 172.21.15.103:0/2242434213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21c0074f40 msgr2=0x7f21c019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21bf400640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21c0074f40 0x7f21c019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21bf400640 1 -- 172.21.15.103:0/2242434213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21c010e210 msgr2=0x7f21c019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21bf400640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21c010e210 0x7f21c019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:55.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.381+0000 7f21bf400640 1 -- 172.21.15.103:0/2242434213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c019f5c0 con 0x7f21c0074620 2024-09-16T02:03:55.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.382+0000 7f21bf400640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c0074620 0x7f21c019a610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f21b000b4a0 tx=0x7f21b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:55.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.382+0000 7f21b7e00640 1 -- 172.21.15.103:0/2242434213 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0014070 con 0x7f21c0074620 2024-09-16T02:03:55.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.382+0000 7f21b7e00640 1 -- 172.21.15.103:0/2242434213 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21b0010af0 con 0x7f21c0074620 2024-09-16T02:03:55.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.382+0000 7f21c5200640 1 -- 172.21.15.103:0/2242434213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c019f8b0 con 0x7f21c0074620 2024-09-16T02:03:55.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.382+0000 7f21b7e00640 1 -- 172.21.15.103:0/2242434213 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0017d10 con 0x7f21c0074620 2024-09-16T02:03:55.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.382+0000 7f21c5200640 1 -- 172.21.15.103:0/2242434213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c01a7190 con 0x7f21c0074620 2024-09-16T02:03:55.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.383+0000 7f21c5200640 1 -- 172.21.15.103:0/2242434213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c0074a80 con 0x7f21c0074620 2024-09-16T02:03:55.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.385+0000 7f21b7e00640 1 -- 172.21.15.103:0/2242434213 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f21b0017070 con 0x7f21c0074620 2024-09-16T02:03:55.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.387+0000 7f21b7e00640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f21a40a32d0 0x7f21a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:55.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.387+0000 7f21b7e00640 1 -- 172.21.15.103:0/2242434213 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f21b00cec00 con 0x7f21c0074620 2024-09-16T02:03:55.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.387+0000 7f21bea00640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f21a40a32d0 0x7f21a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:55.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.388+0000 7f21bea00640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f21a40a32d0 0x7f21a40a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000c00 tx=0x7f21ac005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:55.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.391+0000 7f21b7e00640 1 -- 172.21.15.103:0/2242434213 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21b009b330 con 0x7f21c0074620 2024-09-16T02:03:55.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.598+0000 7f21c5200640 1 -- 172.21.15.103:0/2242434213 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f21c0062630 con 0x7f21a40a32d0 2024-09-16T02:03:55.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.603+0000 7f21b7e00640 1 -- 172.21.15.103:0/2242434213 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f21c0062630 con 0x7f21a40a32d0 2024-09-16T02:03:55.605 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_0/a344c68f-6ac9-45cd-acbf-3f9fee02e807 2024-09-16T02:03:55.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.605+0000 7f21c5200640 1 -- 172.21.15.103:0/2242434213 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f21a40a32d0 msgr2=0x7f21a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:55.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.605+0000 7f21c5200640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f21a40a32d0 0x7f21a40a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000c00 tx=0x7f21ac005df0 comp rx=0 tx=0).stop 2024-09-16T02:03:55.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.605+0000 7f21c5200640 1 -- 172.21.15.103:0/2242434213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c0074620 msgr2=0x7f21c019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:55.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.605+0000 7f21c5200640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c0074620 0x7f21c019a610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f21b000b4a0 tx=0x7f21b000b960 comp rx=0 tx=0).stop 2024-09-16T02:03:55.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.605+0000 7f21c5200640 1 -- 172.21.15.103:0/2242434213 shutdown_connections 2024-09-16T02:03:55.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.605+0000 7f21c5200640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f21a40a32d0 0x7f21a40a5790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:55.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.605+0000 7f21c5200640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21c010e210 0x7f21c019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:55.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.605+0000 7f21c5200640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21c0074f40 0x7f21c019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:55.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.605+0000 7f21c5200640 1 --2- 172.21.15.103:0/2242434213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c0074620 0x7f21c019a610 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:55.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.605+0000 7f21c5200640 1 -- 172.21.15.103:0/2242434213 >> 172.21.15.103:0/2242434213 conn(0x7f21c00fd710 msgr2=0x7f21c00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:55.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.606+0000 7f21c5200640 1 -- 172.21.15.103:0/2242434213 shutdown_connections 2024-09-16T02:03:55.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:55.606+0000 7f21c5200640 1 -- 172.21.15.103:0/2242434213 wait complete. 2024-09-16T02:03:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:55 smithi172 ceph-mon[35340]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:55 smithi172 ceph-mon[35340]: pgmap v290: 97 pgs: 97 active+clean; 589 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-16T02:03:55.925 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:55 smithi103 ceph-mon[30398]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:55.925 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:55 smithi103 ceph-mon[30398]: pgmap v290: 97 pgs: 97 active+clean; 589 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-16T02:03:56.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:55 smithi136 ceph-mon[35323]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:56.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:55 smithi136 ceph-mon[35323]: pgmap v290: 97 pgs: 97 active+clean; 589 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-16T02:03:56.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-16T02:03:56.629 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:56.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:56 smithi172 ceph-mon[35340]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:56.812 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:56 smithi103 ceph-mon[30398]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:57.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:56 smithi136 ceph-mon[35323]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:57.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.154+0000 7f06c3200640 1 -- 172.21.15.103:0/2330693161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c4102370 msgr2=0x7f06c410e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.154+0000 7f06c3200640 1 --2- 172.21.15.103:0/2330693161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c4102370 0x7f06c410e8a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009860 tx=0x7f06bc02a340 comp rx=0 tx=0).stop 2024-09-16T02:03:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.154+0000 7f06c3200640 1 -- 172.21.15.103:0/2330693161 shutdown_connections 2024-09-16T02:03:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.154+0000 7f06c3200640 1 --2- 172.21.15.103:0/2330693161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c4102370 0x7f06c410e8a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.154+0000 7f06c3200640 1 --2- 172.21.15.103:0/2330693161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c41019d0 0x7f06c4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.154+0000 7f06c3200640 1 --2- 172.21.15.103:0/2330693161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c41079d0 0x7f06c4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.154+0000 7f06c3200640 1 -- 172.21.15.103:0/2330693161 >> 172.21.15.103:0/2330693161 conn(0x7f06c40fd710 msgr2=0x7f06c40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.154+0000 7f06c3200640 1 -- 172.21.15.103:0/2330693161 shutdown_connections 2024-09-16T02:03:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.154+0000 7f06c3200640 1 -- 172.21.15.103:0/2330693161 wait complete. 2024-09-16T02:03:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.155+0000 7f06c3200640 1 Processor -- start 2024-09-16T02:03:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.155+0000 7f06c3200640 1 -- start start 2024-09-16T02:03:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.155+0000 7f06c3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c41019d0 0x7f06c419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.155+0000 7f06c3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4102370 0x7f06c419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06c3200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c41079d0 0x7f06c419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06c3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4111dd0 con 0x7f06c4102370 2024-09-16T02:03:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06c3200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4111c50 con 0x7f06c41079d0 2024-09-16T02:03:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06c3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4111f50 con 0x7f06c41019d0 2024-09-16T02:03:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06b9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4102370 0x7f06c419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06b9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4102370 0x7f06c419a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49372/0 (socket says 172.21.15.103:49372) 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06b9400640 1 -- 172.21.15.103:0/1215321560 learned_addr learned my addr 172.21.15.103:0/1215321560 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06c1e00640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c41019d0 0x7f06c419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06c2800640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c41079d0 0x7f06c419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06b9400640 1 -- 172.21.15.103:0/1215321560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c41019d0 msgr2=0x7f06c419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06b9400640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c41019d0 0x7f06c419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06b9400640 1 -- 172.21.15.103:0/1215321560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c41079d0 msgr2=0x7f06c419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06b9400640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c41079d0 0x7f06c419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.156+0000 7f06b9400640 1 -- 172.21.15.103:0/1215321560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c419f3a0 con 0x7f06c4102370 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.157+0000 7f06b9400640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4102370 0x7f06c419a930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00e9c0 tx=0x7f06ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.157+0000 7f06bb400640 1 -- 172.21.15.103:0/1215321560 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac00cde0 con 0x7f06c4102370 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.157+0000 7f06bb400640 1 -- 172.21.15.103:0/1215321560 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06ac0035f0 con 0x7f06c4102370 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.157+0000 7f06c3200640 1 -- 172.21.15.103:0/1215321560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c419f690 con 0x7f06c4102370 2024-09-16T02:03:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.157+0000 7f06bb400640 1 -- 172.21.15.103:0/1215321560 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac010690 con 0x7f06c4102370 2024-09-16T02:03:57.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.157+0000 7f06c3200640 1 -- 172.21.15.103:0/1215321560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c41a6f70 con 0x7f06c4102370 2024-09-16T02:03:57.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.158+0000 7f06c3200640 1 -- 172.21.15.103:0/1215321560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c4101e30 con 0x7f06c4102370 2024-09-16T02:03:57.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.160+0000 7f06bb400640 1 -- 172.21.15.103:0/1215321560 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f06ac0037a0 con 0x7f06c4102370 2024-09-16T02:03:57.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.161+0000 7f06bb400640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06a80a32d0 0x7f06a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:57.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.161+0000 7f06bb400640 1 -- 172.21.15.103:0/1215321560 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f06ac0cfce0 con 0x7f06c4102370 2024-09-16T02:03:57.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.162+0000 7f06c1e00640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06a80a32d0 0x7f06a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:57.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.162+0000 7f06c1e00640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06a80a32d0 0x7f06a80a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005fd0 tx=0x7f06b4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:57.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.165+0000 7f06bb400640 1 -- 172.21.15.103:0/1215321560 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f06ac09c950 con 0x7f06c4102370 2024-09-16T02:03:57.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.364+0000 7f06c3200640 1 -- 172.21.15.103:0/1215321560 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f06c40fec30 con 0x7f06a80a32d0 2024-09-16T02:03:57.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.394+0000 7f06bb400640 1 -- 172.21.15.103:0/1215321560 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f06c40fec30 con 0x7f06a80a32d0 2024-09-16T02:03:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 -- 172.21.15.103:0/1215321560 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06a80a32d0 msgr2=0x7f06a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06a80a32d0 0x7f06a80a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005fd0 tx=0x7f06b4009500 comp rx=0 tx=0).stop 2024-09-16T02:03:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 -- 172.21.15.103:0/1215321560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4102370 msgr2=0x7f06c419a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4102370 0x7f06c419a930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00e9c0 tx=0x7f06ac00ee80 comp rx=0 tx=0).stop 2024-09-16T02:03:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 -- 172.21.15.103:0/1215321560 shutdown_connections 2024-09-16T02:03:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f06a80a32d0 0x7f06a80a5790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c41079d0 0x7f06c419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4102370 0x7f06c419a930 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 --2- 172.21.15.103:0/1215321560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c41019d0 0x7f06c419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 -- 172.21.15.103:0/1215321560 >> 172.21.15.103:0/1215321560 conn(0x7f06c40fd710 msgr2=0x7f06c410d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 -- 172.21.15.103:0/1215321560 shutdown_connections 2024-09-16T02:03:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:57.397+0000 7f06c3200640 1 -- 172.21.15.103:0/1215321560 wait complete. 2024-09-16T02:03:57.703 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:57 smithi103 ceph-mon[30398]: pgmap v291: 97 pgs: 97 active+clean; 589 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-16T02:03:57.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:57 smithi172 ceph-mon[35340]: pgmap v291: 97 pgs: 97 active+clean; 589 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-16T02:03:58.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:57 smithi136 ceph-mon[35323]: pgmap v291: 97 pgs: 97 active+clean; 589 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-16T02:03:58.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-16T02:03:58.537 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:03:58.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:58 smithi103 ceph-mon[30398]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:59.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:58 smithi136 ceph-mon[35323]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:59.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:58 smithi172 ceph-mon[35340]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:03:59.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.246+0000 7fc46f200640 1 -- 172.21.15.103:0/2012356887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc468108f60 msgr2=0x7fc468110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.246+0000 7fc46f200640 1 --2- 172.21.15.103:0/2012356887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc468108f60 0x7fc468110ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc464009ca0 tx=0x7fc46402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.246+0000 7fc46f200640 1 -- 172.21.15.103:0/2012356887 shutdown_connections 2024-09-16T02:03:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.246+0000 7fc46f200640 1 --2- 172.21.15.103:0/2012356887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc468108f60 0x7fc468110ae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.246+0000 7fc46f200640 1 --2- 172.21.15.103:0/2012356887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4681047e0 0x7fc468108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.246+0000 7fc46f200640 1 --2- 172.21.15.103:0/2012356887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc468103e30 0x7fc468104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.246+0000 7fc46f200640 1 -- 172.21.15.103:0/2012356887 >> 172.21.15.103:0/2012356887 conn(0x7fc4680ff910 msgr2=0x7fc468101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.246+0000 7fc46f200640 1 -- 172.21.15.103:0/2012356887 shutdown_connections 2024-09-16T02:03:59.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.246+0000 7fc46f200640 1 -- 172.21.15.103:0/2012356887 wait complete. 2024-09-16T02:03:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.247+0000 7fc46f200640 1 Processor -- start 2024-09-16T02:03:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.247+0000 7fc46f200640 1 -- start start 2024-09-16T02:03:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc468103e30 0x7fc46819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4681047e0 0x7fc46819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc468108f60 0x7fc46819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:59.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc468114010 con 0x7fc468108f60 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc468113e90 con 0x7fc4681047e0 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc468114190 con 0x7fc468103e30 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc468103e30 0x7fc46819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc468103e30 0x7fc46819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:45142/0 (socket says 172.21.15.103:45142) 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46de00640 1 -- 172.21.15.103:0/2994783122 learned_addr learned my addr 172.21.15.103:0/2994783122 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc468108f60 0x7fc46819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46d400640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4681047e0 0x7fc46819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46de00640 1 -- 172.21.15.103:0/2994783122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4681047e0 msgr2=0x7fc46819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46de00640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4681047e0 0x7fc46819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46de00640 1 -- 172.21.15.103:0/2994783122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc468108f60 msgr2=0x7fc46819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46de00640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc468108f60 0x7fc46819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.248+0000 7fc46de00640 1 -- 172.21.15.103:0/2994783122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46819f460 con 0x7fc468103e30 2024-09-16T02:03:59.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.249+0000 7fc46de00640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc468103e30 0x7fc46819a4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00e9f0 tx=0x7fc45c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.249+0000 7fc462a00640 1 -- 172.21.15.103:0/2994783122 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00cd80 con 0x7fc468103e30 2024-09-16T02:03:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.249+0000 7fc46f200640 1 -- 172.21.15.103:0/2994783122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46819f750 con 0x7fc468103e30 2024-09-16T02:03:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.249+0000 7fc462a00640 1 -- 172.21.15.103:0/2994783122 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c010910 con 0x7fc468103e30 2024-09-16T02:03:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.249+0000 7fc462a00640 1 -- 172.21.15.103:0/2994783122 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0188f0 con 0x7fc468103e30 2024-09-16T02:03:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.249+0000 7fc46f200640 1 -- 172.21.15.103:0/2994783122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4681a7030 con 0x7fc468103e30 2024-09-16T02:03:59.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.250+0000 7fc46f200640 1 -- 172.21.15.103:0/2994783122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc468105310 con 0x7fc468103e30 2024-09-16T02:03:59.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.252+0000 7fc462a00640 1 -- 172.21.15.103:0/2994783122 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc45c010ac0 con 0x7fc468103e30 2024-09-16T02:03:59.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.253+0000 7fc462a00640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4500a32d0 0x7fc4500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:03:59.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.254+0000 7fc462a00640 1 -- 172.21.15.103:0/2994783122 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fc45c0d0180 con 0x7fc468103e30 2024-09-16T02:03:59.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.254+0000 7fc46d400640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4500a32d0 0x7fc4500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:03:59.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.254+0000 7fc46d400640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4500a32d0 0x7fc4500a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc46810a430 tx=0x7fc45800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:03:59.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.256+0000 7fc462a00640 1 -- 172.21.15.103:0/2994783122 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc45c09c7a0 con 0x7fc468103e30 2024-09-16T02:03:59.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.457+0000 7fc46f200640 1 -- 172.21.15.103:0/2994783122 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc46810c200 con 0x7fc4500a32d0 2024-09-16T02:03:59.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.462+0000 7fc462a00640 1 -- 172.21.15.103:0/2994783122 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc46810c200 con 0x7fc4500a32d0 2024-09-16T02:03:59.463 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_1/1d178de4-9eb9-4acc-93f7-1d9d1e58235e 2024-09-16T02:03:59.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 -- 172.21.15.103:0/2994783122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4500a32d0 msgr2=0x7fc4500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:59.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4500a32d0 0x7fc4500a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc46810a430 tx=0x7fc45800b390 comp rx=0 tx=0).stop 2024-09-16T02:03:59.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 -- 172.21.15.103:0/2994783122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc468103e30 msgr2=0x7fc46819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:03:59.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc468103e30 0x7fc46819a4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00e9f0 tx=0x7fc45c00eeb0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 -- 172.21.15.103:0/2994783122 shutdown_connections 2024-09-16T02:03:59.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4500a32d0 0x7fc4500a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc468108f60 0x7fc46819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4681047e0 0x7fc46819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 --2- 172.21.15.103:0/2994783122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc468103e30 0x7fc46819a4b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:03:59.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 -- 172.21.15.103:0/2994783122 >> 172.21.15.103:0/2994783122 conn(0x7fc4680ff910 msgr2=0x7fc468100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:03:59.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 -- 172.21.15.103:0/2994783122 shutdown_connections 2024-09-16T02:03:59.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:03:59.465+0000 7fc46f200640 1 -- 172.21.15.103:0/2994783122 wait complete. 2024-09-16T02:03:59.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:03:59 smithi103 ceph-mon[30398]: pgmap v292: 97 pgs: 97 active+clean; 621 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-16T02:04:00.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:03:59 smithi136 ceph-mon[35323]: pgmap v292: 97 pgs: 97 active+clean; 621 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-16T02:04:00.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:03:59 smithi172 ceph-mon[35340]: pgmap v292: 97 pgs: 97 active+clean; 621 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-16T02:04:00.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:04:00.586 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:00.861 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:00 smithi103 ceph-mon[30398]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:04:01.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:00 smithi136 ceph-mon[35323]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:04:01.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:00 smithi172 ceph-mon[35340]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:04:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.113+0000 7fb539200640 1 -- 172.21.15.103:0/3357253691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb534076720 msgr2=0x7fb534076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.113+0000 7fb539200640 1 --2- 172.21.15.103:0/3357253691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb534076720 0x7fb534076b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb524009900 tx=0x7fb52402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.113+0000 7fb539200640 1 -- 172.21.15.103:0/3357253691 shutdown_connections 2024-09-16T02:04:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.113+0000 7fb539200640 1 --2- 172.21.15.103:0/3357253691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb534112870 0x7fb534114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.113+0000 7fb539200640 1 --2- 172.21.15.103:0/3357253691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb534077040 0x7fb534112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.113+0000 7fb539200640 1 --2- 172.21.15.103:0/3357253691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb534076720 0x7fb534076b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.113+0000 7fb539200640 1 -- 172.21.15.103:0/3357253691 >> 172.21.15.103:0/3357253691 conn(0x7fb5340ff8d0 msgr2=0x7fb534101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.113+0000 7fb539200640 1 -- 172.21.15.103:0/3357253691 shutdown_connections 2024-09-16T02:04:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.113+0000 7fb539200640 1 -- 172.21.15.103:0/3357253691 wait complete. 2024-09-16T02:04:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.114+0000 7fb539200640 1 Processor -- start 2024-09-16T02:04:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.114+0000 7fb539200640 1 -- start start 2024-09-16T02:04:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.114+0000 7fb539200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb534076720 0x7fb5341a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.114+0000 7fb539200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb534077040 0x7fb5341a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.114+0000 7fb539200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb534112870 0x7fb53419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb539200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb534118450 con 0x7fb534076720 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb539200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb5341182d0 con 0x7fb534112870 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb539200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5341185d0 con 0x7fb534077040 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb52aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb534077040 0x7fb5341a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb533400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb534076720 0x7fb5341a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb533e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb534112870 0x7fb53419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb52aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb534077040 0x7fb5341a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:45166/0 (socket says 172.21.15.103:45166) 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb52aa00640 1 -- 172.21.15.103:0/3889641913 learned_addr learned my addr 172.21.15.103:0/3889641913 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb52aa00640 1 -- 172.21.15.103:0/3889641913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb534112870 msgr2=0x7fb53419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb52aa00640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb534112870 0x7fb53419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb52aa00640 1 -- 172.21.15.103:0/3889641913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb534076720 msgr2=0x7fb5341a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb52aa00640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb534076720 0x7fb5341a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb52aa00640 1 -- 172.21.15.103:0/3889641913 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53419ef70 con 0x7fb534077040 2024-09-16T02:04:01.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.115+0000 7fb533e00640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb534112870 0x7fb53419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:01.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.116+0000 7fb52aa00640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb534077040 0x7fb5341a4b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e9f0 tx=0x7fb51c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:01.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.116+0000 7fb530c00640 1 -- 172.21.15.103:0/3889641913 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c00cd80 con 0x7fb534077040 2024-09-16T02:04:01.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.116+0000 7fb539200640 1 -- 172.21.15.103:0/3889641913 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53419f260 con 0x7fb534077040 2024-09-16T02:04:01.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.116+0000 7fb530c00640 1 -- 172.21.15.103:0/3889641913 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb51c010910 con 0x7fb534077040 2024-09-16T02:04:01.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.116+0000 7fb530c00640 1 -- 172.21.15.103:0/3889641913 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0188f0 con 0x7fb534077040 2024-09-16T02:04:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.116+0000 7fb539200640 1 -- 172.21.15.103:0/3889641913 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5341ab5b0 con 0x7fb534077040 2024-09-16T02:04:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.117+0000 7fb539200640 1 -- 172.21.15.103:0/3889641913 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb53419f5d0 con 0x7fb534077040 2024-09-16T02:04:01.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.119+0000 7fb530c00640 1 -- 172.21.15.103:0/3889641913 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb51c010ac0 con 0x7fb534077040 2024-09-16T02:04:01.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.120+0000 7fb530c00640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5180a32d0 0x7fb5180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:01.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.120+0000 7fb530c00640 1 -- 172.21.15.103:0/3889641913 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fb51c0d0180 con 0x7fb534077040 2024-09-16T02:04:01.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.121+0000 7fb533400640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5180a32d0 0x7fb5180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:01.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.121+0000 7fb533400640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5180a32d0 0x7fb5180a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb524000c00 tx=0x7fb524005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:01.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.122+0000 7fb530c00640 1 -- 172.21.15.103:0/3889641913 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb51c09c7a0 con 0x7fb534077040 2024-09-16T02:04:01.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.329+0000 7fb539200640 1 -- 172.21.15.103:0/3889641913 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb534077e60 con 0x7fb534077040 2024-09-16T02:04:01.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.330+0000 7fb530c00640 1 -- 172.21.15.103:0/3889641913 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7fb51c0a0aa0 con 0x7fb534077040 2024-09-16T02:04:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:01.331 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":23,"btime":"2024-09-16T02:03:50:098205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:03:49.812580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24505,24506,24545]},"id":1}]} 2024-09-16T02:04:01.331 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 23 2024-09-16T02:04:01.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 -- 172.21.15.103:0/3889641913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5180a32d0 msgr2=0x7fb5180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:01.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5180a32d0 0x7fb5180a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb524000c00 tx=0x7fb524005b50 comp rx=0 tx=0).stop 2024-09-16T02:04:01.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 -- 172.21.15.103:0/3889641913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb534077040 msgr2=0x7fb5341a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:01.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb534077040 0x7fb5341a4b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e9f0 tx=0x7fb51c00eeb0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 -- 172.21.15.103:0/3889641913 shutdown_connections 2024-09-16T02:04:01.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5180a32d0 0x7fb5180a5790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb534112870 0x7fb53419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb534077040 0x7fb5341a4b70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 --2- 172.21.15.103:0/3889641913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb534076720 0x7fb5341a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:01.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 -- 172.21.15.103:0/3889641913 >> 172.21.15.103:0/3889641913 conn(0x7fb5340ff8d0 msgr2=0x7fb534101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:01.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 -- 172.21.15.103:0/3889641913 shutdown_connections 2024-09-16T02:04:01.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:01.332+0000 7fb539200640 1 -- 172.21.15.103:0/3889641913 wait complete. 2024-09-16T02:04:02.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:01 smithi103 ceph-mon[30398]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-16T02:04:02.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:01 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3889641913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:02.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:01 smithi136 ceph-mon[35323]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-16T02:04:02.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:01 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3889641913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:02.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:01 smithi172 ceph-mon[35340]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-16T02:04:02.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:01 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3889641913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:02.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:04:02.444 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:02.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.966+0000 7f4044a00640 1 -- 172.21.15.103:0/1467898831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4040103e30 msgr2=0x7f4040104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:02.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.966+0000 7f4044a00640 1 --2- 172.21.15.103:0/1467898831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4040103e30 0x7f4040104210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4030009ca0 tx=0x7f403002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:04:02.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.966+0000 7f4044a00640 1 -- 172.21.15.103:0/1467898831 shutdown_connections 2024-09-16T02:04:02.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.966+0000 7f4044a00640 1 --2- 172.21.15.103:0/1467898831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4040108f60 0x7f4040110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:02.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.966+0000 7f4044a00640 1 --2- 172.21.15.103:0/1467898831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40401047e0 0x7f4040108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:02.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.966+0000 7f4044a00640 1 --2- 172.21.15.103:0/1467898831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4040103e30 0x7f4040104210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:02.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.966+0000 7f4044a00640 1 -- 172.21.15.103:0/1467898831 >> 172.21.15.103:0/1467898831 conn(0x7f40400ff8d0 msgr2=0x7f4040101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:02.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.966+0000 7f4044a00640 1 -- 172.21.15.103:0/1467898831 shutdown_connections 2024-09-16T02:04:02.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.966+0000 7f4044a00640 1 -- 172.21.15.103:0/1467898831 wait complete. 2024-09-16T02:04:02.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.967+0000 7f4044a00640 1 Processor -- start 2024-09-16T02:04:02.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.967+0000 7f4044a00640 1 -- start start 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.967+0000 7f4044a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4040103e30 0x7f404019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f4044a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40401047e0 0x7f404019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f4044a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4040108f60 0x7f404019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f4044a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4040113fc0 con 0x7f4040108f60 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f4044a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4040113e40 con 0x7f4040103e30 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f4044a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4040114140 con 0x7f40401047e0 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4040103e30 0x7f404019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4040108f60 0x7f404019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4040103e30 0x7f404019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:47642/0 (socket says 172.21.15.103:47642) 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403e800640 1 -- 172.21.15.103:0/3942369995 learned_addr learned my addr 172.21.15.103:0/3942369995 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403de00640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40401047e0 0x7f404019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403f200640 1 -- 172.21.15.103:0/3942369995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40401047e0 msgr2=0x7f404019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403f200640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40401047e0 0x7f404019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:02.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403f200640 1 -- 172.21.15.103:0/3942369995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4040103e30 msgr2=0x7f404019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:02.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403f200640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4040103e30 0x7f404019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:02.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403f200640 1 -- 172.21.15.103:0/3942369995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404019f430 con 0x7f4040108f60 2024-09-16T02:04:02.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.968+0000 7f403e800640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4040103e30 0x7f404019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:02.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.969+0000 7f403f200640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4040108f60 0x7f404019ed50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f403800d6b0 tx=0x7f403800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:02.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.969+0000 7f4037400640 1 -- 172.21.15.103:0/3942369995 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038017070 con 0x7f4040108f60 2024-09-16T02:04:02.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.969+0000 7f4044a00640 1 -- 172.21.15.103:0/3942369995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404019f720 con 0x7f4040108f60 2024-09-16T02:04:02.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.969+0000 7f4037400640 1 -- 172.21.15.103:0/3942369995 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4038009d00 con 0x7f4040108f60 2024-09-16T02:04:02.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.969+0000 7f4037400640 1 -- 172.21.15.103:0/3942369995 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038003d60 con 0x7f4040108f60 2024-09-16T02:04:02.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.969+0000 7f4044a00640 1 -- 172.21.15.103:0/3942369995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40401a7000 con 0x7f4040108f60 2024-09-16T02:04:02.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.970+0000 7f4044a00640 1 -- 172.21.15.103:0/3942369995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4040104210 con 0x7f4040108f60 2024-09-16T02:04:02.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.972+0000 7f4037400640 1 -- 172.21.15.103:0/3942369995 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4038009820 con 0x7f4040108f60 2024-09-16T02:04:02.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.973+0000 7f4037400640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f40240a3350 0x7f40240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:02.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.973+0000 7f4037400640 1 -- 172.21.15.103:0/3942369995 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f40380cf1a0 con 0x7f4040108f60 2024-09-16T02:04:02.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.974+0000 7f403e800640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f40240a3350 0x7f40240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:02.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.974+0000 7f403e800640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f40240a3350 0x7f40240a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4030031f80 tx=0x7f4030004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:02.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:02.978+0000 7f4037400640 1 -- 172.21.15.103:0/3942369995 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f403809b7c0 con 0x7f4040108f60 2024-09-16T02:04:03.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.156+0000 7f4044a00640 1 -- 172.21.15.103:0/3942369995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f404010bbb0 con 0x7f4040108f60 2024-09-16T02:04:03.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.157+0000 7f4037400640 1 -- 172.21.15.103:0/3942369995 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22361 (secure 0 0 0) 0x7f403809fac0 con 0x7f4040108f60 2024-09-16T02:04:03.159 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:03.159 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":77,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:03:37.830327+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:04:03.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 -- 172.21.15.103:0/3942369995 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f40240a3350 msgr2=0x7f40240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:03.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f40240a3350 0x7f40240a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4030031f80 tx=0x7f4030004a40 comp rx=0 tx=0).stop 2024-09-16T02:04:03.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 -- 172.21.15.103:0/3942369995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4040108f60 msgr2=0x7f404019ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:03.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4040108f60 0x7f404019ed50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f403800d6b0 tx=0x7f403800db70 comp rx=0 tx=0).stop 2024-09-16T02:04:03.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 -- 172.21.15.103:0/3942369995 shutdown_connections 2024-09-16T02:04:03.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f40240a3350 0x7f40240a5810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:03.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4040108f60 0x7f404019ed50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:03.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40401047e0 0x7f404019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:03.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 --2- 172.21.15.103:0/3942369995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4040103e30 0x7f404019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:03.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 -- 172.21.15.103:0/3942369995 >> 172.21.15.103:0/3942369995 conn(0x7f40400ff8d0 msgr2=0x7f4040100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:03.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 -- 172.21.15.103:0/3942369995 shutdown_connections 2024-09-16T02:04:03.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:03.161+0000 7f4044a00640 1 -- 172.21.15.103:0/3942369995 wait complete. 2024-09-16T02:04:03.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:03 smithi103 ceph-mon[30398]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-16T02:04:03.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:03 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3942369995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:03.977 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T02:04:03.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:04:04.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:03 smithi136 ceph-mon[35323]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-16T02:04:04.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:03 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3942369995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:04.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:03 smithi172 ceph-mon[35340]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-16T02:04:04.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:03 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3942369995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:04.347 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:04.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.884+0000 7f3431400640 1 -- 172.21.15.103:0/1696773262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f342c074f40 msgr2=0x7f342c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:04.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.884+0000 7f3431400640 1 --2- 172.21.15.103:0/1696773262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f342c074f40 0x7f342c10de80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3418009900 tx=0x7f341802a4a0 comp rx=0 tx=0).stop 2024-09-16T02:04:04.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.885+0000 7f3431400640 1 -- 172.21.15.103:0/1696773262 shutdown_connections 2024-09-16T02:04:04.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.885+0000 7f3431400640 1 --2- 172.21.15.103:0/1696773262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c10e3c0 0x7f342c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:04.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.885+0000 7f3431400640 1 --2- 172.21.15.103:0/1696773262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f342c074f40 0x7f342c10de80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:04.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.885+0000 7f3431400640 1 --2- 172.21.15.103:0/1696773262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f342c074620 0x7f342c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:04.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.885+0000 7f3431400640 1 -- 172.21.15.103:0/1696773262 >> 172.21.15.103:0/1696773262 conn(0x7f342c0fd8c0 msgr2=0x7f342c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:04.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.885+0000 7f3431400640 1 -- 172.21.15.103:0/1696773262 shutdown_connections 2024-09-16T02:04:04.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.885+0000 7f3431400640 1 -- 172.21.15.103:0/1696773262 wait complete. 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f3431400640 1 Processor -- start 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f3431400640 1 -- start start 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f3431400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f342c074620 0x7f342c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f3431400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f342c074f40 0x7f342c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f3431400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c10e3c0 0x7f342c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f3431400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f342c114220 con 0x7f342c074620 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f3431400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1140a0 con 0x7f342c10e3c0 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f3431400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1143a0 con 0x7f342c074f40 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f342a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f342c074620 0x7f342c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f342a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f342c074620 0x7f342c19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38462/0 (socket says 172.21.15.103:38462) 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.886+0000 7f342a800640 1 -- 172.21.15.103:0/685189558 learned_addr learned my addr 172.21.15.103:0/685189558 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:04.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.887+0000 7f342b200640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c10e3c0 0x7f342c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.887+0000 7f3429e00640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f342c074f40 0x7f342c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.887+0000 7f342a800640 1 -- 172.21.15.103:0/685189558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f342c074f40 msgr2=0x7f342c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.887+0000 7f342a800640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f342c074f40 0x7f342c19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.887+0000 7f342a800640 1 -- 172.21.15.103:0/685189558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c10e3c0 msgr2=0x7f342c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.887+0000 7f342a800640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c10e3c0 0x7f342c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.887+0000 7f342a800640 1 -- 172.21.15.103:0/685189558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c19f6d0 con 0x7f342c074620 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.887+0000 7f342a800640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f342c074620 0x7f342c19a720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d950 tx=0x7f341c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.887+0000 7f3423400640 1 -- 172.21.15.103:0/685189558 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c017070 con 0x7f342c074620 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.888+0000 7f3423400640 1 -- 172.21.15.103:0/685189558 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f341c009c30 con 0x7f342c074620 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.888+0000 7f3423400640 1 -- 172.21.15.103:0/685189558 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c00f6b0 con 0x7f342c074620 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.888+0000 7f3431400640 1 -- 172.21.15.103:0/685189558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c19f9c0 con 0x7f342c074620 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.888+0000 7f3431400640 1 -- 172.21.15.103:0/685189558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a72a0 con 0x7f342c074620 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.889+0000 7f3431400640 1 -- 172.21.15.103:0/685189558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c074a00 con 0x7f342c074620 2024-09-16T02:04:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.890+0000 7f3423400640 1 -- 172.21.15.103:0/685189558 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f341c0109d0 con 0x7f342c074620 2024-09-16T02:04:04.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.892+0000 7f3423400640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f34100a32d0 0x7f34100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:04.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.892+0000 7f3423400640 1 -- 172.21.15.103:0/685189558 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f341c0cf5d0 con 0x7f342c074620 2024-09-16T02:04:04.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.892+0000 7f3429e00640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f34100a32d0 0x7f34100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:04.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.893+0000 7f3429e00640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f34100a32d0 0x7f34100a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3418000c00 tx=0x7f3418005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:04.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:04.896+0000 7f3423400640 1 -- 172.21.15.103:0/685189558 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f341c009de0 con 0x7f342c074620 2024-09-16T02:04:05.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.069+0000 7f3431400640 1 -- 172.21.15.103:0/685189558 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f342c062630 con 0x7f34100a32d0 2024-09-16T02:04:05.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.077+0000 7f3423400640 1 -- 172.21.15.103:0/685189558 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258700 (secure 0 0 0) 0x7f342c062630 con 0x7f34100a32d0 2024-09-16T02:04:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:05.082 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 -- 172.21.15.103:0/685189558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f34100a32d0 msgr2=0x7f34100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f34100a32d0 0x7f34100a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3418000c00 tx=0x7f3418005b50 comp rx=0 tx=0).stop 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 -- 172.21.15.103:0/685189558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f342c074620 msgr2=0x7f342c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f342c074620 0x7f342c19a720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d950 tx=0x7f341c00de10 comp rx=0 tx=0).stop 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 -- 172.21.15.103:0/685189558 shutdown_connections 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f34100a32d0 0x7f34100a5790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c10e3c0 0x7f342c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f342c074f40 0x7f342c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 --2- 172.21.15.103:0/685189558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f342c074620 0x7f342c19a720 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d950 tx=0x7f341c00de10 comp rx=0 tx=0).stop 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 -- 172.21.15.103:0/685189558 >> 172.21.15.103:0/685189558 conn(0x7f342c0fd8c0 msgr2=0x7f342c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 -- 172.21.15.103:0/685189558 shutdown_connections 2024-09-16T02:04:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:05.081+0000 7f3431400640 1 -- 172.21.15.103:0/685189558 wait complete. 2024-09-16T02:04:05.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:05 smithi103 ceph-mon[30398]: pgmap v295: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-09-16T02:04:05.799 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-16T02:04:04.453976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645110,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":241,"num_read_kb":223,"num_write":337,"num_write_kb":3037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369448,"kb_used_data":11880,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572376,"statfs":{"total":1151940427776,"available":1151562113024,"internally_reserved":0,"allocated":12165120,"data_stored":6789451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49520,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":31,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003140"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048779+0000","last_change":"2024-09-16T02:03:37.050873+0000","last_active":"2024-09-16T02:03:38.048779+0000","last_peered":"2024-09-16T02:03:38.048779+0000","last_clean":"2024-09-16T02:03:38.048779+0000","last_became_active":"2024-09-16T02:03:37.050202+0000","last_became_peered":"2024-09-16T02:03:37.050202+0000","last_unstale":"2024-09-16T02:03:38.048779+0000","last_undegraded":"2024-09-16T02:03:38.048779+0000","last_fullsized":"2024-09-16T02:03:38.048779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:12.971199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035219+0000","last_change":"2024-09-16T02:03:37.048034+0000","last_active":"2024-09-16T02:03:38.035219+0000","last_peered":"2024-09-16T02:03:38.035219+0000","last_clean":"2024-09-16T02:03:38.035219+0000","last_became_active":"2024-09-16T02:03:37.047216+0000","last_became_peered":"2024-09-16T02:03:37.047216+0000","last_unstale":"2024-09-16T02:03:38.035219+0000","last_undegraded":"2024-09-16T02:03:38.035219+0000","last_fullsized":"2024-09-16T02:03:38.035219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:48.580466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035331+0000","last_change":"2024-09-16T02:03:37.059287+0000","last_active":"2024-09-16T02:03:38.035331+0000","last_peered":"2024-09-16T02:03:38.035331+0000","last_clean":"2024-09-16T02:03:38.035331+0000","last_became_active":"2024-09-16T02:03:37.055270+0000","last_became_peered":"2024-09-16T02:03:37.055270+0000","last_unstale":"2024-09-16T02:03:38.035331+0000","last_undegraded":"2024-09-16T02:03:38.035331+0000","last_fullsized":"2024-09-16T02:03:38.035331+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:29.430514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046356+0000","last_change":"2024-09-16T02:03:37.052473+0000","last_active":"2024-09-16T02:03:38.046356+0000","last_peered":"2024-09-16T02:03:38.046356+0000","last_clean":"2024-09-16T02:03:38.046356+0000","last_became_active":"2024-09-16T02:03:37.052015+0000","last_became_peered":"2024-09-16T02:03:37.052015+0000","last_unstale":"2024-09-16T02:03:38.046356+0000","last_undegraded":"2024-09-16T02:03:38.046356+0000","last_fullsized":"2024-09-16T02:03:38.046356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:03.964132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.033262+0000","last_change":"2024-09-16T02:03:37.051516+0000","last_active":"2024-09-16T02:03:38.033262+0000","last_peered":"2024-09-16T02:03:38.033262+0000","last_clean":"2024-09-16T02:03:38.033262+0000","last_became_active":"2024-09-16T02:03:37.051201+0000","last_became_peered":"2024-09-16T02:03:37.051201+0000","last_unstale":"2024-09-16T02:03:38.033262+0000","last_undegraded":"2024-09-16T02:03:38.033262+0000","last_fullsized":"2024-09-16T02:03:38.033262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:45.327477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047424+0000","last_change":"2024-09-16T02:03:37.048912+0000","last_active":"2024-09-16T02:03:38.047424+0000","last_peered":"2024-09-16T02:03:38.047424+0000","last_clean":"2024-09-16T02:03:38.047424+0000","last_became_active":"2024-09-16T02:03:37.048577+0000","last_became_peered":"2024-09-16T02:03:37.048577+0000","last_unstale":"2024-09-16T02:03:38.047424+0000","last_undegraded":"2024-09-16T02:03:38.047424+0000","last_fullsized":"2024-09-16T02:03:38.047424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:25.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045070+0000","last_change":"2024-09-16T02:03:37.057427+0000","last_active":"2024-09-16T02:03:38.045070+0000","last_peered":"2024-09-16T02:03:38.045070+0000","last_clean":"2024-09-16T02:03:38.045070+0000","last_became_active":"2024-09-16T02:03:37.057111+0000","last_became_peered":"2024-09-16T02:03:37.057111+0000","last_unstale":"2024-09-16T02:03:38.045070+0000","last_undegraded":"2024-09-16T02:03:38.045070+0000","last_fullsized":"2024-09-16T02:03:38.045070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:44.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046741+0000","last_change":"2024-09-16T02:03:37.051085+0000","last_active":"2024-09-16T02:03:38.046741+0000","last_peered":"2024-09-16T02:03:38.046741+0000","last_clean":"2024-09-16T02:03:38.046741+0000","last_became_active":"2024-09-16T02:03:37.050790+0000","last_became_peered":"2024-09-16T02:03:37.050790+0000","last_unstale":"2024-09-16T02:03:38.046741+0000","last_undegraded":"2024-09-16T02:03:38.046741+0000","last_fullsized":"2024-09-16T02:03:38.046741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:18.620901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035545+0000","last_change":"2024-09-16T02:03:37.054258+0000","last_active":"2024-09-16T02:03:38.035545+0000","last_peered":"2024-09-16T02:03:38.035545+0000","last_clean":"2024-09-16T02:03:38.035545+0000","last_became_active":"2024-09-16T02:03:37.053939+0000","last_became_peered":"2024-09-16T02:03:37.053939+0000","last_unstale":"2024-09-16T02:03:38.035545+0000","last_undegraded":"2024-09-16T02:03:38.035545+0000","last_fullsized":"2024-09-16T02:03:38.035545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:11.076147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048385+0000","last_change":"2024-09-16T02:03:37.049828+0000","last_active":"2024-09-16T02:03:38.048385+0000","last_peered":"2024-09-16T02:03:38.048385+0000","last_clean":"2024-09-16T02:03:38.048385+0000","last_became_active":"2024-09-16T02:03:37.049577+0000","last_became_peered":"2024-09-16T02:03:37.049577+0000","last_unstale":"2024-09-16T02:03:38.048385+0000","last_undegraded":"2024-09-16T02:03:38.048385+0000","last_fullsized":"2024-09-16T02:03:38.048385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:37.176431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046251+0000","last_change":"2024-09-16T02:03:37.053268+0000","last_active":"2024-09-16T02:03:38.046251+0000","last_peered":"2024-09-16T02:03:38.046251+0000","last_clean":"2024-09-16T02:03:38.046251+0000","last_became_active":"2024-09-16T02:03:37.052933+0000","last_became_peered":"2024-09-16T02:03:37.052933+0000","last_unstale":"2024-09-16T02:03:38.046251+0000","last_undegraded":"2024-09-16T02:03:38.046251+0000","last_fullsized":"2024-09-16T02:03:38.046251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:32.003728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046516+0000","last_change":"2024-09-16T02:03:37.046339+0000","last_active":"2024-09-16T02:03:38.046516+0000","last_peered":"2024-09-16T02:03:38.046516+0000","last_clean":"2024-09-16T02:03:38.046516+0000","last_became_active":"2024-09-16T02:03:37.043262+0000","last_became_peered":"2024-09-16T02:03:37.043262+0000","last_unstale":"2024-09-16T02:03:38.046516+0000","last_undegraded":"2024-09-16T02:03:38.046516+0000","last_fullsized":"2024-09-16T02:03:38.046516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:58.614672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048257+0000","last_change":"2024-09-16T02:03:37.043856+0000","last_active":"2024-09-16T02:03:38.048257+0000","last_peered":"2024-09-16T02:03:38.048257+0000","last_clean":"2024-09-16T02:03:38.048257+0000","last_became_active":"2024-09-16T02:03:37.043546+0000","last_became_peered":"2024-09-16T02:03:37.043546+0000","last_unstale":"2024-09-16T02:03:38.048257+0000","last_undegraded":"2024-09-16T02:03:38.048257+0000","last_fullsized":"2024-09-16T02:03:38.048257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:57:07.213575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035119+0000","last_change":"2024-09-16T02:03:37.048844+0000","last_active":"2024-09-16T02:03:38.035119+0000","last_peered":"2024-09-16T02:03:38.035119+0000","last_clean":"2024-09-16T02:03:38.035119+0000","last_became_active":"2024-09-16T02:03:37.048522+0000","last_became_peered":"2024-09-16T02:03:37.048522+0000","last_unstale":"2024-09-16T02:03:38.035119+0000","last_undegraded":"2024-09-16T02:03:38.035119+0000","last_fullsized":"2024-09-16T02:03:38.035119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:50.759847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035028+0000","last_change":"2024-09-16T02:03:37.043236+0000","last_active":"2024-09-16T02:03:38.035028+0000","last_peered":"2024-09-16T02:03:38.035028+0000","last_clean":"2024-09-16T02:03:38.035028+0000","last_became_active":"2024-09-16T02:03:37.042967+0000","last_became_peered":"2024-09-16T02:03:37.042967+0000","last_unstale":"2024-09-16T02:03:38.035028+0000","last_undegraded":"2024-09-16T02:03:38.035028+0000","last_fullsized":"2024-09-16T02:03:38.035028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:02.739678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035121+0000","last_change":"2024-09-16T02:03:37.052972+0000","last_active":"2024-09-16T02:03:38.035121+0000","last_peered":"2024-09-16T02:03:38.035121+0000","last_clean":"2024-09-16T02:03:38.035121+0000","last_became_active":"2024-09-16T02:03:37.052665+0000","last_became_peered":"2024-09-16T02:03:37.052665+0000","last_unstale":"2024-09-16T02:03:38.035121+0000","last_undegraded":"2024-09-16T02:03:38.035121+0000","last_fullsized":"2024-09-16T02:03:38.035121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:59.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048818+0000","last_change":"2024-09-16T02:03:37.051203+0000","last_active":"2024-09-16T02:03:38.048818+0000","last_peered":"2024-09-16T02:03:38.048818+0000","last_clean":"2024-09-16T02:03:38.048818+0000","last_became_active":"2024-09-16T02:03:37.049828+0000","last_became_peered":"2024-09-16T02:03:37.049828+0000","last_unstale":"2024-09-16T02:03:38.048818+0000","last_undegraded":"2024-09-16T02:03:38.048818+0000","last_fullsized":"2024-09-16T02:03:38.048818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:47.566594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048163+0000","last_change":"2024-09-16T02:03:37.056903+0000","last_active":"2024-09-16T02:03:38.048163+0000","last_peered":"2024-09-16T02:03:38.048163+0000","last_clean":"2024-09-16T02:03:38.048163+0000","last_became_active":"2024-09-16T02:03:37.056579+0000","last_became_peered":"2024-09-16T02:03:37.056579+0000","last_unstale":"2024-09-16T02:03:38.048163+0000","last_undegraded":"2024-09-16T02:03:38.048163+0000","last_fullsized":"2024-09-16T02:03:38.048163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:23.652294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043469+0000","last_change":"2024-09-16T02:03:37.054413+0000","last_active":"2024-09-16T02:03:38.043469+0000","last_peered":"2024-09-16T02:03:38.043469+0000","last_clean":"2024-09-16T02:03:38.043469+0000","last_became_active":"2024-09-16T02:03:37.054040+0000","last_became_peered":"2024-09-16T02:03:37.054040+0000","last_unstale":"2024-09-16T02:03:38.043469+0000","last_undegraded":"2024-09-16T02:03:38.043469+0000","last_fullsized":"2024-09-16T02:03:38.043469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:04.517313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048372+0000","last_change":"2024-09-16T02:03:37.053454+0000","last_active":"2024-09-16T02:03:38.048372+0000","last_peered":"2024-09-16T02:03:38.048372+0000","last_clean":"2024-09-16T02:03:38.048372+0000","last_became_active":"2024-09-16T02:03:37.053123+0000","last_became_peered":"2024-09-16T02:03:37.053123+0000","last_unstale":"2024-09-16T02:03:38.048372+0000","last_undegraded":"2024-09-16T02:03:38.048372+0000","last_fullsized":"2024-09-16T02:03:38.048372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:52.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048111+0000","last_change":"2024-09-16T02:03:37.042772+0000","last_active":"2024-09-16T02:03:38.048111+0000","last_peered":"2024-09-16T02:03:38.048111+0000","last_clean":"2024-09-16T02:03:38.048111+0000","last_became_active":"2024-09-16T02:03:37.041832+0000","last_became_peered":"2024-09-16T02:03:37.041832+0000","last_unstale":"2024-09-16T02:03:38.048111+0000","last_undegraded":"2024-09-16T02:03:38.048111+0000","last_fullsized":"2024-09-16T02:03:38.048111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:54.436291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035219+0000","last_change":"2024-09-16T02:03:37.058484+0000","last_active":"2024-09-16T02:03:38.035219+0000","last_peered":"2024-09-16T02:03:38.035219+0000","last_clean":"2024-09-16T02:03:38.035219+0000","last_became_active":"2024-09-16T02:03:37.058047+0000","last_became_peered":"2024-09-16T02:03:37.058047+0000","last_unstale":"2024-09-16T02:03:38.035219+0000","last_undegraded":"2024-09-16T02:03:38.035219+0000","last_fullsized":"2024-09-16T02:03:38.035219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:23.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047461+0000","last_change":"2024-09-16T02:03:37.058576+0000","last_active":"2024-09-16T02:03:38.047461+0000","last_peered":"2024-09-16T02:03:38.047461+0000","last_clean":"2024-09-16T02:03:38.047461+0000","last_became_active":"2024-09-16T02:03:37.057624+0000","last_became_peered":"2024-09-16T02:03:37.057624+0000","last_unstale":"2024-09-16T02:03:38.047461+0000","last_undegraded":"2024-09-16T02:03:38.047461+0000","last_fullsized":"2024-09-16T02:03:38.047461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:21.929461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043131+0000","last_change":"2024-09-16T02:03:37.050889+0000","last_active":"2024-09-16T02:03:38.043131+0000","last_peered":"2024-09-16T02:03:38.043131+0000","last_clean":"2024-09-16T02:03:38.043131+0000","last_became_active":"2024-09-16T02:03:37.050535+0000","last_became_peered":"2024-09-16T02:03:37.050535+0000","last_unstale":"2024-09-16T02:03:38.043131+0000","last_undegraded":"2024-09-16T02:03:38.043131+0000","last_fullsized":"2024-09-16T02:03:38.043131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:46.955962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048934+0000","last_change":"2024-09-16T02:03:37.053865+0000","last_active":"2024-09-16T02:03:38.048934+0000","last_peered":"2024-09-16T02:03:38.048934+0000","last_clean":"2024-09-16T02:03:38.048934+0000","last_became_active":"2024-09-16T02:03:37.053563+0000","last_became_peered":"2024-09-16T02:03:37.053563+0000","last_unstale":"2024-09-16T02:03:38.048934+0000","last_undegraded":"2024-09-16T02:03:38.048934+0000","last_fullsized":"2024-09-16T02:03:38.048934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:24.109471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.042208+0000","last_change":"2024-09-16T02:03:37.054681+0000","last_active":"2024-09-16T02:03:38.042208+0000","last_peered":"2024-09-16T02:03:38.042208+0000","last_clean":"2024-09-16T02:03:38.042208+0000","last_became_active":"2024-09-16T02:03:37.054324+0000","last_became_peered":"2024-09-16T02:03:37.054324+0000","last_unstale":"2024-09-16T02:03:38.042208+0000","last_undegraded":"2024-09-16T02:03:38.042208+0000","last_fullsized":"2024-09-16T02:03:38.042208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:20.472520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048558+0000","last_change":"2024-09-16T02:03:37.047713+0000","last_active":"2024-09-16T02:03:38.048558+0000","last_peered":"2024-09-16T02:03:38.048558+0000","last_clean":"2024-09-16T02:03:38.048558+0000","last_became_active":"2024-09-16T02:03:37.047353+0000","last_became_peered":"2024-09-16T02:03:37.047353+0000","last_unstale":"2024-09-16T02:03:38.048558+0000","last_undegraded":"2024-09-16T02:03:38.048558+0000","last_fullsized":"2024-09-16T02:03:38.048558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:05.868909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047693+0000","last_change":"2024-09-16T02:03:37.038931+0000","last_active":"2024-09-16T02:03:38.047693+0000","last_peered":"2024-09-16T02:03:38.047693+0000","last_clean":"2024-09-16T02:03:38.047693+0000","last_became_active":"2024-09-16T02:03:37.037234+0000","last_became_peered":"2024-09-16T02:03:37.037234+0000","last_unstale":"2024-09-16T02:03:38.047693+0000","last_undegraded":"2024-09-16T02:03:38.047693+0000","last_fullsized":"2024-09-16T02:03:38.047693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:54.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035239+0000","last_change":"2024-09-16T02:03:37.051104+0000","last_active":"2024-09-16T02:03:38.035239+0000","last_peered":"2024-09-16T02:03:38.035239+0000","last_clean":"2024-09-16T02:03:38.035239+0000","last_became_active":"2024-09-16T02:03:37.048668+0000","last_became_peered":"2024-09-16T02:03:37.048668+0000","last_unstale":"2024-09-16T02:03:38.035239+0000","last_undegraded":"2024-09-16T02:03:38.035239+0000","last_fullsized":"2024-09-16T02:03:38.035239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:03.113818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.041767+0000","last_change":"2024-09-16T02:03:37.057688+0000","last_active":"2024-09-16T02:03:38.041767+0000","last_peered":"2024-09-16T02:03:38.041767+0000","last_clean":"2024-09-16T02:03:38.041767+0000","last_became_active":"2024-09-16T02:03:37.057347+0000","last_became_peered":"2024-09-16T02:03:37.057347+0000","last_unstale":"2024-09-16T02:03:38.041767+0000","last_undegraded":"2024-09-16T02:03:38.041767+0000","last_fullsized":"2024-09-16T02:03:38.041767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:30.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.042078+0000","last_change":"2024-09-16T02:03:37.054689+0000","last_active":"2024-09-16T02:03:38.042078+0000","last_peered":"2024-09-16T02:03:38.042078+0000","last_clean":"2024-09-16T02:03:38.042078+0000","last_became_active":"2024-09-16T02:03:37.054335+0000","last_became_peered":"2024-09-16T02:03:37.054335+0000","last_unstale":"2024-09-16T02:03:38.042078+0000","last_undegraded":"2024-09-16T02:03:38.042078+0000","last_fullsized":"2024-09-16T02:03:38.042078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:03.389582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048239+0000","last_change":"2024-09-16T02:03:37.043596+0000","last_active":"2024-09-16T02:03:38.048239+0000","last_peered":"2024-09-16T02:03:38.048239+0000","last_clean":"2024-09-16T02:03:38.048239+0000","last_became_active":"2024-09-16T02:03:37.043261+0000","last_became_peered":"2024-09-16T02:03:37.043261+0000","last_unstale":"2024-09-16T02:03:38.048239+0000","last_undegraded":"2024-09-16T02:03:38.048239+0000","last_fullsized":"2024-09-16T02:03:38.048239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:03.833126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.033285+0000","last_change":"2024-09-16T02:03:37.043018+0000","last_active":"2024-09-16T02:03:38.033285+0000","last_peered":"2024-09-16T02:03:38.033285+0000","last_clean":"2024-09-16T02:03:38.033285+0000","last_became_active":"2024-09-16T02:03:37.042437+0000","last_became_peered":"2024-09-16T02:03:37.042437+0000","last_unstale":"2024-09-16T02:03:38.033285+0000","last_undegraded":"2024-09-16T02:03:38.033285+0000","last_fullsized":"2024-09-16T02:03:38.033285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.193852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:04:01.123783+0000","last_change":"2024-09-16T02:03:34.741690+0000","last_active":"2024-09-16T02:04:01.123783+0000","last_peered":"2024-09-16T02:04:01.123783+0000","last_clean":"2024-09-16T02:04:01.123783+0000","last_became_active":"2024-09-16T02:03:34.739546+0000","last_became_peered":"2024-09-16T02:03:34.739546+0000","last_unstale":"2024-09-16T02:04:01.123783+0000","last_undegraded":"2024-09-16T02:04:01.123783+0000","last_fullsized":"2024-09-16T02:04:01.123783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:38.402329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043463+0000","last_change":"2024-09-16T02:03:37.052664+0000","last_active":"2024-09-16T02:03:38.043463+0000","last_peered":"2024-09-16T02:03:38.043463+0000","last_clean":"2024-09-16T02:03:38.043463+0000","last_became_active":"2024-09-16T02:03:37.051310+0000","last_became_peered":"2024-09-16T02:03:37.051310+0000","last_unstale":"2024-09-16T02:03:38.043463+0000","last_undegraded":"2024-09-16T02:03:38.043463+0000","last_fullsized":"2024-09-16T02:03:38.043463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:57.979053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.099870+0000","last_change":"2024-09-16T02:03:34.738020+0000","last_active":"2024-09-16T02:03:47.099870+0000","last_peered":"2024-09-16T02:03:47.099870+0000","last_clean":"2024-09-16T02:03:47.099870+0000","last_became_active":"2024-09-16T02:03:34.737644+0000","last_became_peered":"2024-09-16T02:03:34.737644+0000","last_unstale":"2024-09-16T02:03:47.099870+0000","last_undegraded":"2024-09-16T02:03:47.099870+0000","last_fullsized":"2024-09-16T02:03:47.099870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:56.249707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.100080+0000","last_change":"2024-09-16T02:03:34.745325+0000","last_active":"2024-09-16T02:03:47.100080+0000","last_peered":"2024-09-16T02:03:47.100080+0000","last_clean":"2024-09-16T02:03:47.100080+0000","last_became_active":"2024-09-16T02:03:34.744730+0000","last_became_peered":"2024-09-16T02:03:34.744730+0000","last_unstale":"2024-09-16T02:03:47.100080+0000","last_undegraded":"2024-09-16T02:03:47.100080+0000","last_fullsized":"2024-09-16T02:03:47.100080+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046876+0000","last_change":"2024-09-16T02:03:37.058973+0000","last_active":"2024-09-16T02:03:38.046876+0000","last_peered":"2024-09-16T02:03:38.046876+0000","last_clean":"2024-09-16T02:03:38.046876+0000","last_became_active":"2024-09-16T02:03:37.058685+0000","last_became_peered":"2024-09-16T02:03:37.058685+0000","last_unstale":"2024-09-16T02:03:38.046876+0000","last_undegraded":"2024-09-16T02:03:38.046876+0000","last_fullsized":"2024-09-16T02:03:38.046876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:46.030440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043011+0000","last_change":"2024-09-16T02:03:34.737842+0000","last_active":"2024-09-16T02:03:38.043011+0000","last_peered":"2024-09-16T02:03:38.043011+0000","last_clean":"2024-09-16T02:03:38.043011+0000","last_became_active":"2024-09-16T02:03:34.737391+0000","last_became_peered":"2024-09-16T02:03:34.737391+0000","last_unstale":"2024-09-16T02:03:38.043011+0000","last_undegraded":"2024-09-16T02:03:38.043011+0000","last_fullsized":"2024-09-16T02:03:38.043011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:39.214180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047367+0000","last_change":"2024-09-16T02:03:37.049705+0000","last_active":"2024-09-16T02:03:38.047367+0000","last_peered":"2024-09-16T02:03:38.047367+0000","last_clean":"2024-09-16T02:03:38.047367+0000","last_became_active":"2024-09-16T02:03:37.049392+0000","last_became_peered":"2024-09-16T02:03:37.049392+0000","last_unstale":"2024-09-16T02:03:38.047367+0000","last_undegraded":"2024-09-16T02:03:38.047367+0000","last_fullsized":"2024-09-16T02:03:38.047367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.932391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:04:01.123199+0000","last_change":"2024-09-16T02:03:34.742268+0000","last_active":"2024-09-16T02:04:01.123199+0000","last_peered":"2024-09-16T02:04:01.123199+0000","last_clean":"2024-09-16T02:04:01.123199+0000","last_became_active":"2024-09-16T02:03:34.741934+0000","last_became_peered":"2024-09-16T02:03:34.741934+0000","last_unstale":"2024-09-16T02:04:01.123199+0000","last_undegraded":"2024-09-16T02:04:01.123199+0000","last_fullsized":"2024-09-16T02:04:01.123199+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:30.301552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035453+0000","last_change":"2024-09-16T02:03:37.059119+0000","last_active":"2024-09-16T02:03:38.035453+0000","last_peered":"2024-09-16T02:03:38.035453+0000","last_clean":"2024-09-16T02:03:38.035453+0000","last_became_active":"2024-09-16T02:03:37.058738+0000","last_became_peered":"2024-09-16T02:03:37.058738+0000","last_unstale":"2024-09-16T02:03:38.035453+0000","last_undegraded":"2024-09-16T02:03:38.035453+0000","last_fullsized":"2024-09-16T02:03:38.035453+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.496053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.089084+0000","last_change":"2024-09-16T02:03:34.738291+0000","last_active":"2024-09-16T02:03:46.089084+0000","last_peered":"2024-09-16T02:03:46.089084+0000","last_clean":"2024-09-16T02:03:46.089084+0000","last_became_active":"2024-09-16T02:03:34.737436+0000","last_became_peered":"2024-09-16T02:03:34.737436+0000","last_unstale":"2024-09-16T02:03:46.089084+0000","last_undegraded":"2024-09-16T02:03:46.089084+0000","last_fullsized":"2024-09-16T02:03:46.089084+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:13.925417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035504+0000","last_change":"2024-09-16T02:03:37.050349+0000","last_active":"2024-09-16T02:03:38.035504+0000","last_peered":"2024-09-16T02:03:38.035504+0000","last_clean":"2024-09-16T02:03:38.035504+0000","last_became_active":"2024-09-16T02:03:37.049978+0000","last_became_peered":"2024-09-16T02:03:37.049978+0000","last_unstale":"2024-09-16T02:03:38.035504+0000","last_undegraded":"2024-09-16T02:03:38.035504+0000","last_fullsized":"2024-09-16T02:03:38.035504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:45.119683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.089329+0000","last_change":"2024-09-16T02:03:34.740246+0000","last_active":"2024-09-16T02:03:46.089329+0000","last_peered":"2024-09-16T02:03:46.089329+0000","last_clean":"2024-09-16T02:03:46.089329+0000","last_became_active":"2024-09-16T02:03:34.739908+0000","last_became_peered":"2024-09-16T02:03:34.739908+0000","last_unstale":"2024-09-16T02:03:46.089329+0000","last_undegraded":"2024-09-16T02:03:46.089329+0000","last_fullsized":"2024-09-16T02:03:46.089329+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:35.299733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035464+0000","last_change":"2024-09-16T02:03:37.056562+0000","last_active":"2024-09-16T02:03:38.035464+0000","last_peered":"2024-09-16T02:03:38.035464+0000","last_clean":"2024-09-16T02:03:38.035464+0000","last_became_active":"2024-09-16T02:03:37.056226+0000","last_became_peered":"2024-09-16T02:03:37.056226+0000","last_unstale":"2024-09-16T02:03:38.035464+0000","last_undegraded":"2024-09-16T02:03:38.035464+0000","last_fullsized":"2024-09-16T02:03:38.035464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.741538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'25","reported_seq":80,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:04:02.126557+0000","last_change":"2024-09-16T02:03:34.736403+0000","last_active":"2024-09-16T02:04:02.126557+0000","last_peered":"2024-09-16T02:04:02.126557+0000","last_clean":"2024-09-16T02:04:02.126557+0000","last_became_active":"2024-09-16T02:03:34.736074+0000","last_became_peered":"2024-09-16T02:03:34.736074+0000","last_unstale":"2024-09-16T02:04:02.126557+0000","last_undegraded":"2024-09-16T02:04:02.126557+0000","last_fullsized":"2024-09-16T02:04:02.126557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:34.769862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":66,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.033652+0000","last_change":"2024-09-16T02:03:37.046803+0000","last_active":"2024-09-16T02:03:38.033652+0000","last_peered":"2024-09-16T02:03:38.033652+0000","last_clean":"2024-09-16T02:03:38.033652+0000","last_became_active":"2024-09-16T02:03:37.046459+0000","last_became_peered":"2024-09-16T02:03:37.046459+0000","last_unstale":"2024-09-16T02:03:38.033652+0000","last_undegraded":"2024-09-16T02:03:38.033652+0000","last_fullsized":"2024-09-16T02:03:38.033652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:55:38.460795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.099275+0000","last_change":"2024-09-16T02:03:34.739016+0000","last_active":"2024-09-16T02:03:47.099275+0000","last_peered":"2024-09-16T02:03:47.099275+0000","last_clean":"2024-09-16T02:03:47.099275+0000","last_became_active":"2024-09-16T02:03:34.738583+0000","last_became_peered":"2024-09-16T02:03:34.738583+0000","last_unstale":"2024-09-16T02:03:47.099275+0000","last_undegraded":"2024-09-16T02:03:47.099275+0000","last_fullsized":"2024-09-16T02:03:47.099275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:08.255832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046158+0000","last_change":"2024-09-16T02:03:37.053353+0000","last_active":"2024-09-16T02:03:38.046158+0000","last_peered":"2024-09-16T02:03:38.046158+0000","last_clean":"2024-09-16T02:03:38.046158+0000","last_became_active":"2024-09-16T02:03:37.051140+0000","last_became_peered":"2024-09-16T02:03:37.051140+0000","last_unstale":"2024-09-16T02:03:38.046158+0000","last_undegraded":"2024-09-16T02:03:38.046158+0000","last_fullsized":"2024-09-16T02:03:38.046158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:34.636338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.089256+0000","last_change":"2024-09-16T02:03:34.742111+0000","last_active":"2024-09-16T02:03:46.089256+0000","last_peered":"2024-09-16T02:03:46.089256+0000","last_clean":"2024-09-16T02:03:46.089256+0000","last_became_active":"2024-09-16T02:03:34.741816+0000","last_became_peered":"2024-09-16T02:03:34.741816+0000","last_unstale":"2024-09-16T02:03:46.089256+0000","last_undegraded":"2024-09-16T02:03:46.089256+0000","last_fullsized":"2024-09-16T02:03:46.089256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:36.861995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046067+0000","last_change":"2024-09-16T02:03:37.047152+0000","last_active":"2024-09-16T02:03:38.046067+0000","last_peered":"2024-09-16T02:03:38.046067+0000","last_clean":"2024-09-16T02:03:38.046067+0000","last_became_active":"2024-09-16T02:03:37.046675+0000","last_became_peered":"2024-09-16T02:03:37.046675+0000","last_unstale":"2024-09-16T02:03:38.046067+0000","last_undegraded":"2024-09-16T02:03:38.046067+0000","last_fullsized":"2024-09-16T02:03:38.046067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:58.236519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:04:01.123220+0000","last_change":"2024-09-16T02:03:34.738487+0000","last_active":"2024-09-16T02:04:01.123220+0000","last_peered":"2024-09-16T02:04:01.123220+0000","last_clean":"2024-09-16T02:04:01.123220+0000","last_became_active":"2024-09-16T02:03:34.737061+0000","last_became_peered":"2024-09-16T02:03:34.737061+0000","last_unstale":"2024-09-16T02:04:01.123220+0000","last_undegraded":"2024-09-16T02:04:01.123220+0000","last_fullsized":"2024-09-16T02:04:01.123220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:24.231640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045125+0000","last_change":"2024-09-16T02:03:37.044260+0000","last_active":"2024-09-16T02:03:38.045125+0000","last_peered":"2024-09-16T02:03:38.045125+0000","last_clean":"2024-09-16T02:03:38.045125+0000","last_became_active":"2024-09-16T02:03:37.043932+0000","last_became_peered":"2024-09-16T02:03:37.043932+0000","last_unstale":"2024-09-16T02:03:38.045125+0000","last_undegraded":"2024-09-16T02:03:38.045125+0000","last_fullsized":"2024-09-16T02:03:38.045125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:55.718155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.100668+0000","last_change":"2024-09-16T02:03:34.742451+0000","last_active":"2024-09-16T02:03:47.100668+0000","last_peered":"2024-09-16T02:03:47.100668+0000","last_clean":"2024-09-16T02:03:47.100668+0000","last_became_active":"2024-09-16T02:03:34.742089+0000","last_became_peered":"2024-09-16T02:03:34.742089+0000","last_unstale":"2024-09-16T02:03:47.100668+0000","last_undegraded":"2024-09-16T02:03:47.100668+0000","last_fullsized":"2024-09-16T02:03:47.100668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:20.426059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043675+0000","last_change":"2024-09-16T02:03:37.053040+0000","last_active":"2024-09-16T02:03:38.043675+0000","last_peered":"2024-09-16T02:03:38.043675+0000","last_clean":"2024-09-16T02:03:38.043675+0000","last_became_active":"2024-09-16T02:03:37.052185+0000","last_became_peered":"2024-09-16T02:03:37.052185+0000","last_unstale":"2024-09-16T02:03:38.043675+0000","last_undegraded":"2024-09-16T02:03:38.043675+0000","last_fullsized":"2024-09-16T02:03:38.043675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:27.676678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:48.094783+0000","last_change":"2024-09-16T02:03:34.740234+0000","last_active":"2024-09-16T02:03:48.094783+0000","last_peered":"2024-09-16T02:03:48.094783+0000","last_clean":"2024-09-16T02:03:48.094783+0000","last_became_active":"2024-09-16T02:03:34.739905+0000","last_became_peered":"2024-09-16T02:03:34.739905+0000","last_unstale":"2024-09-16T02:03:48.094783+0000","last_undegraded":"2024-09-16T02:03:48.094783+0000","last_fullsized":"2024-09-16T02:03:48.094783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:19:02.347369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048636+0000","last_change":"2024-09-16T02:03:37.049420+0000","last_active":"2024-09-16T02:03:38.048636+0000","last_peered":"2024-09-16T02:03:38.048636+0000","last_clean":"2024-09-16T02:03:38.048636+0000","last_became_active":"2024-09-16T02:03:37.047671+0000","last_became_peered":"2024-09-16T02:03:37.047671+0000","last_unstale":"2024-09-16T02:03:38.048636+0000","last_undegraded":"2024-09-16T02:03:38.048636+0000","last_fullsized":"2024-09-16T02:03:38.048636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:23.023791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.050781+0000","last_change":"2024-09-16T02:03:34.738516+0000","last_active":"2024-09-16T02:03:38.050781+0000","last_peered":"2024-09-16T02:03:38.050781+0000","last_clean":"2024-09-16T02:03:38.050781+0000","last_became_active":"2024-09-16T02:03:34.737701+0000","last_became_peered":"2024-09-16T02:03:34.737701+0000","last_unstale":"2024-09-16T02:03:38.050781+0000","last_undegraded":"2024-09-16T02:03:38.050781+0000","last_fullsized":"2024-09-16T02:03:38.050781+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:42.101865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045953+0000","last_change":"2024-09-16T02:03:37.046820+0000","last_active":"2024-09-16T02:03:38.045953+0000","last_peered":"2024-09-16T02:03:38.045953+0000","last_clean":"2024-09-16T02:03:38.045953+0000","last_became_active":"2024-09-16T02:03:37.046082+0000","last_became_peered":"2024-09-16T02:03:37.046082+0000","last_unstale":"2024-09-16T02:03:38.045953+0000","last_undegraded":"2024-09-16T02:03:38.045953+0000","last_fullsized":"2024-09-16T02:03:38.045953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:21.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046593+0000","last_change":"2024-09-16T02:03:34.741908+0000","last_active":"2024-09-16T02:03:38.046593+0000","last_peered":"2024-09-16T02:03:38.046593+0000","last_clean":"2024-09-16T02:03:38.046593+0000","last_became_active":"2024-09-16T02:03:34.739826+0000","last_became_peered":"2024-09-16T02:03:34.739826+0000","last_unstale":"2024-09-16T02:03:38.046593+0000","last_undegraded":"2024-09-16T02:03:38.046593+0000","last_fullsized":"2024-09-16T02:03:38.046593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:04.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:40.502622+0000","last_change":"2024-09-16T02:03:37.056459+0000","last_active":"2024-09-16T02:03:40.502622+0000","last_peered":"2024-09-16T02:03:40.502622+0000","last_clean":"2024-09-16T02:03:40.502622+0000","last_became_active":"2024-09-16T02:03:37.055902+0000","last_became_peered":"2024-09-16T02:03:37.055902+0000","last_unstale":"2024-09-16T02:03:40.502622+0000","last_undegraded":"2024-09-16T02:03:40.502622+0000","last_fullsized":"2024-09-16T02:03:40.502622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:48:30.815654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.100170+0000","last_change":"2024-09-16T02:03:34.741207+0000","last_active":"2024-09-16T02:03:47.100170+0000","last_peered":"2024-09-16T02:03:47.100170+0000","last_clean":"2024-09-16T02:03:47.100170+0000","last_became_active":"2024-09-16T02:03:34.740523+0000","last_became_peered":"2024-09-16T02:03:34.740523+0000","last_unstale":"2024-09-16T02:03:47.100170+0000","last_undegraded":"2024-09-16T02:03:47.100170+0000","last_fullsized":"2024-09-16T02:03:47.100170+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:36.199332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046291+0000","last_change":"2024-09-16T02:03:37.052792+0000","last_active":"2024-09-16T02:03:38.046291+0000","last_peered":"2024-09-16T02:03:38.046291+0000","last_clean":"2024-09-16T02:03:38.046291+0000","last_became_active":"2024-09-16T02:03:37.052301+0000","last_became_peered":"2024-09-16T02:03:37.052301+0000","last_unstale":"2024-09-16T02:03:38.046291+0000","last_undegraded":"2024-09-16T02:03:38.046291+0000","last_fullsized":"2024-09-16T02:03:38.046291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:55.197481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":329,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045809+0000","last_change":"2024-09-16T02:02:21.095893+0000","last_active":"2024-09-16T02:03:38.045809+0000","last_peered":"2024-09-16T02:03:38.045809+0000","last_clean":"2024-09-16T02:03:38.045809+0000","last_became_active":"2024-09-16T02:02:21.095395+0000","last_became_peered":"2024-09-16T02:02:21.095395+0000","last_unstale":"2024-09-16T02:03:38.045809+0000","last_undegraded":"2024-09-16T02:03:38.045809+0000","last_fullsized":"2024-09-16T02:03:38.045809+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_clean_scrub_stamp":"2024-09-16T01:58:50.954254+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:40.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:48.100136+0000","last_change":"2024-09-16T02:03:34.746154+0000","last_active":"2024-09-16T02:03:48.100136+0000","last_peered":"2024-09-16T02:03:48.100136+0000","last_clean":"2024-09-16T02:03:48.100136+0000","last_became_active":"2024-09-16T02:03:34.745646+0000","last_became_peered":"2024-09-16T02:03:34.745646+0000","last_unstale":"2024-09-16T02:03:48.100136+0000","last_undegraded":"2024-09-16T02:03:48.100136+0000","last_fullsized":"2024-09-16T02:03:48.100136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:06.790000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047338+0000","last_change":"2024-09-16T02:03:37.056638+0000","last_active":"2024-09-16T02:03:38.047338+0000","last_peered":"2024-09-16T02:03:38.047338+0000","last_clean":"2024-09-16T02:03:38.047338+0000","last_became_active":"2024-09-16T02:03:37.056043+0000","last_became_peered":"2024-09-16T02:03:37.056043+0000","last_unstale":"2024-09-16T02:03:38.047338+0000","last_undegraded":"2024-09-16T02:03:38.047338+0000","last_fullsized":"2024-09-16T02:03:38.047338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:35:48.353380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.098205+0000","last_change":"2024-09-16T02:03:34.741453+0000","last_active":"2024-09-16T02:03:47.098205+0000","last_peered":"2024-09-16T02:03:47.098205+0000","last_clean":"2024-09-16T02:03:47.098205+0000","last_became_active":"2024-09-16T02:03:34.741128+0000","last_became_peered":"2024-09-16T02:03:34.741128+0000","last_unstale":"2024-09-16T02:03:47.098205+0000","last_undegraded":"2024-09-16T02:03:47.098205+0000","last_fullsized":"2024-09-16T02:03:47.098205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:41.544575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045992+0000","last_change":"2024-09-16T02:03:37.050895+0000","last_active":"2024-09-16T02:03:38.045992+0000","last_peered":"2024-09-16T02:03:38.045992+0000","last_clean":"2024-09-16T02:03:38.045992+0000","last_became_active":"2024-09-16T02:03:37.050559+0000","last_became_peered":"2024-09-16T02:03:37.050559+0000","last_unstale":"2024-09-16T02:03:38.045992+0000","last_undegraded":"2024-09-16T02:03:38.045992+0000","last_fullsized":"2024-09-16T02:03:38.045992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:20.067954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:59.119888+0000","last_change":"2024-09-16T02:03:34.740286+0000","last_active":"2024-09-16T02:03:59.119888+0000","last_peered":"2024-09-16T02:03:59.119888+0000","last_clean":"2024-09-16T02:03:59.119888+0000","last_became_active":"2024-09-16T02:03:34.739892+0000","last_became_peered":"2024-09-16T02:03:34.739892+0000","last_unstale":"2024-09-16T02:03:59.119888+0000","last_undegraded":"2024-09-16T02:03:59.119888+0000","last_fullsized":"2024-09-16T02:03:59.119888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:59:14.809991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.042775+0000","last_change":"2024-09-16T02:03:37.042207+0000","last_active":"2024-09-16T02:03:38.042775+0000","last_peered":"2024-09-16T02:03:38.042775+0000","last_clean":"2024-09-16T02:03:38.042775+0000","last_became_active":"2024-09-16T02:03:37.041283+0000","last_became_peered":"2024-09-16T02:03:37.041283+0000","last_unstale":"2024-09-16T02:03:38.042775+0000","last_undegraded":"2024-09-16T02:03:38.042775+0000","last_fullsized":"2024-09-16T02:03:38.042775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:45.569721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045742+0000","last_change":"2024-09-16T02:03:34.737834+0000","last_active":"2024-09-16T02:03:38.045742+0000","last_peered":"2024-09-16T02:03:38.045742+0000","last_clean":"2024-09-16T02:03:38.045742+0000","last_became_active":"2024-09-16T02:03:34.736706+0000","last_became_peered":"2024-09-16T02:03:34.736706+0000","last_unstale":"2024-09-16T02:03:38.045742+0000","last_undegraded":"2024-09-16T02:03:38.045742+0000","last_fullsized":"2024-09-16T02:03:38.045742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:27.037351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047783+0000","last_change":"2024-09-16T02:03:37.060888+0000","last_active":"2024-09-16T02:03:38.047783+0000","last_peered":"2024-09-16T02:03:38.047783+0000","last_clean":"2024-09-16T02:03:38.047783+0000","last_became_active":"2024-09-16T02:03:37.060458+0000","last_became_peered":"2024-09-16T02:03:37.060458+0000","last_unstale":"2024-09-16T02:03:38.047783+0000","last_undegraded":"2024-09-16T02:03:38.047783+0000","last_fullsized":"2024-09-16T02:03:38.047783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:37.425376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047887+0000","last_change":"2024-09-16T02:03:34.742298+0000","last_active":"2024-09-16T02:03:38.047887+0000","last_peered":"2024-09-16T02:03:38.047887+0000","last_clean":"2024-09-16T02:03:38.047887+0000","last_became_active":"2024-09-16T02:03:34.741960+0000","last_became_peered":"2024-09-16T02:03:34.741960+0000","last_unstale":"2024-09-16T02:03:38.047887+0000","last_undegraded":"2024-09-16T02:03:38.047887+0000","last_fullsized":"2024-09-16T02:03:38.047887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:12.832207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047039+0000","last_change":"2024-09-16T02:03:37.060553+0000","last_active":"2024-09-16T02:03:38.047039+0000","last_peered":"2024-09-16T02:03:38.047039+0000","last_clean":"2024-09-16T02:03:38.047039+0000","last_became_active":"2024-09-16T02:03:37.059961+0000","last_became_peered":"2024-09-16T02:03:37.059961+0000","last_unstale":"2024-09-16T02:03:38.047039+0000","last_undegraded":"2024-09-16T02:03:38.047039+0000","last_fullsized":"2024-09-16T02:03:38.047039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:22.904190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.087975+0000","last_change":"2024-09-16T02:03:34.747828+0000","last_active":"2024-09-16T02:03:46.087975+0000","last_peered":"2024-09-16T02:03:46.087975+0000","last_clean":"2024-09-16T02:03:46.087975+0000","last_became_active":"2024-09-16T02:03:34.747548+0000","last_became_peered":"2024-09-16T02:03:34.747548+0000","last_unstale":"2024-09-16T02:03:46.087975+0000","last_undegraded":"2024-09-16T02:03:46.087975+0000","last_fullsized":"2024-09-16T02:03:46.087975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:49.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047552+0000","last_change":"2024-09-16T02:03:37.051477+0000","last_active":"2024-09-16T02:03:38.047552+0000","last_peered":"2024-09-16T02:03:38.047552+0000","last_clean":"2024-09-16T02:03:38.047552+0000","last_became_active":"2024-09-16T02:03:37.048063+0000","last_became_peered":"2024-09-16T02:03:37.048063+0000","last_unstale":"2024-09-16T02:03:38.047552+0000","last_undegraded":"2024-09-16T02:03:38.047552+0000","last_fullsized":"2024-09-16T02:03:38.047552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:15.973972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:58.040008+0000","last_change":"2024-09-16T02:03:34.742935+0000","last_active":"2024-09-16T02:03:58.040008+0000","last_peered":"2024-09-16T02:03:58.040008+0000","last_clean":"2024-09-16T02:03:58.040008+0000","last_became_active":"2024-09-16T02:03:34.742611+0000","last_became_peered":"2024-09-16T02:03:34.742611+0000","last_unstale":"2024-09-16T02:03:58.040008+0000","last_undegraded":"2024-09-16T02:03:58.040008+0000","last_fullsized":"2024-09-16T02:03:58.040008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:36.695220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.049294+0000","last_change":"2024-09-16T02:03:37.054926+0000","last_active":"2024-09-16T02:03:38.049294+0000","last_peered":"2024-09-16T02:03:38.049294+0000","last_clean":"2024-09-16T02:03:38.049294+0000","last_became_active":"2024-09-16T02:03:37.054483+0000","last_became_peered":"2024-09-16T02:03:37.054483+0000","last_unstale":"2024-09-16T02:03:38.049294+0000","last_undegraded":"2024-09-16T02:03:38.049294+0000","last_fullsized":"2024-09-16T02:03:38.049294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.090767+0000","last_change":"2024-09-16T02:03:34.739611+0000","last_active":"2024-09-16T02:03:46.090767+0000","last_peered":"2024-09-16T02:03:46.090767+0000","last_clean":"2024-09-16T02:03:46.090767+0000","last_became_active":"2024-09-16T02:03:34.738090+0000","last_became_peered":"2024-09-16T02:03:34.738090+0000","last_unstale":"2024-09-16T02:03:46.090767+0000","last_undegraded":"2024-09-16T02:03:46.090767+0000","last_fullsized":"2024-09-16T02:03:46.090767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:47.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035368+0000","last_change":"2024-09-16T02:03:37.053020+0000","last_active":"2024-09-16T02:03:38.035368+0000","last_peered":"2024-09-16T02:03:38.035368+0000","last_clean":"2024-09-16T02:03:38.035368+0000","last_became_active":"2024-09-16T02:03:37.052365+0000","last_became_peered":"2024-09-16T02:03:37.052365+0000","last_unstale":"2024-09-16T02:03:38.035368+0000","last_undegraded":"2024-09-16T02:03:38.035368+0000","last_fullsized":"2024-09-16T02:03:38.035368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:54.965028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.089414+0000","last_change":"2024-09-16T02:03:34.740726+0000","last_active":"2024-09-16T02:03:46.089414+0000","last_peered":"2024-09-16T02:03:46.089414+0000","last_clean":"2024-09-16T02:03:46.089414+0000","last_became_active":"2024-09-16T02:03:34.740402+0000","last_became_peered":"2024-09-16T02:03:34.740402+0000","last_unstale":"2024-09-16T02:03:46.089414+0000","last_undegraded":"2024-09-16T02:03:46.089414+0000","last_fullsized":"2024-09-16T02:03:46.089414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:05.761646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.033218+0000","last_change":"2024-09-16T02:03:37.044862+0000","last_active":"2024-09-16T02:03:38.033218+0000","last_peered":"2024-09-16T02:03:38.033218+0000","last_clean":"2024-09-16T02:03:38.033218+0000","last_became_active":"2024-09-16T02:03:37.044406+0000","last_became_peered":"2024-09-16T02:03:37.044406+0000","last_unstale":"2024-09-16T02:03:38.033218+0000","last_undegraded":"2024-09-16T02:03:38.033218+0000","last_fullsized":"2024-09-16T02:03:38.033218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:03.326380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.092552+0000","last_change":"2024-09-16T02:03:34.745505+0000","last_active":"2024-09-16T02:03:46.092552+0000","last_peered":"2024-09-16T02:03:46.092552+0000","last_clean":"2024-09-16T02:03:46.092552+0000","last_became_active":"2024-09-16T02:03:34.745151+0000","last_became_peered":"2024-09-16T02:03:34.745151+0000","last_unstale":"2024-09-16T02:03:46.092552+0000","last_undegraded":"2024-09-16T02:03:46.092552+0000","last_fullsized":"2024-09-16T02:03:46.092552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:57.401176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048300+0000","last_change":"2024-09-16T02:03:37.046692+0000","last_active":"2024-09-16T02:03:38.048300+0000","last_peered":"2024-09-16T02:03:38.048300+0000","last_clean":"2024-09-16T02:03:38.048300+0000","last_became_active":"2024-09-16T02:03:37.046302+0000","last_became_peered":"2024-09-16T02:03:37.046302+0000","last_unstale":"2024-09-16T02:03:38.048300+0000","last_undegraded":"2024-09-16T02:03:38.048300+0000","last_fullsized":"2024-09-16T02:03:38.048300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:47.727431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.041918+0000","last_change":"2024-09-16T02:03:34.739847+0000","last_active":"2024-09-16T02:03:38.041918+0000","last_peered":"2024-09-16T02:03:38.041918+0000","last_clean":"2024-09-16T02:03:38.041918+0000","last_became_active":"2024-09-16T02:03:34.739514+0000","last_became_peered":"2024-09-16T02:03:34.739514+0000","last_unstale":"2024-09-16T02:03:38.041918+0000","last_undegraded":"2024-09-16T02:03:38.041918+0000","last_fullsized":"2024-09-16T02:03:38.041918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:44.617127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047337+0000","last_change":"2024-09-16T02:03:37.054910+0000","last_active":"2024-09-16T02:03:38.047337+0000","last_peered":"2024-09-16T02:03:38.047337+0000","last_clean":"2024-09-16T02:03:38.047337+0000","last_became_active":"2024-09-16T02:03:37.053852+0000","last_became_peered":"2024-09-16T02:03:37.053852+0000","last_unstale":"2024-09-16T02:03:38.047337+0000","last_undegraded":"2024-09-16T02:03:38.047337+0000","last_fullsized":"2024-09-16T02:03:38.047337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:29.042154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.089808+0000","last_change":"2024-09-16T02:03:34.743027+0000","last_active":"2024-09-16T02:03:46.089808+0000","last_peered":"2024-09-16T02:03:46.089808+0000","last_clean":"2024-09-16T02:03:46.089808+0000","last_became_active":"2024-09-16T02:03:34.742658+0000","last_became_peered":"2024-09-16T02:03:34.742658+0000","last_unstale":"2024-09-16T02:03:46.089808+0000","last_undegraded":"2024-09-16T02:03:46.089808+0000","last_fullsized":"2024-09-16T02:03:46.089808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:36.779568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045513+0000","last_change":"2024-09-16T02:03:37.052783+0000","last_active":"2024-09-16T02:03:38.045513+0000","last_peered":"2024-09-16T02:03:38.045513+0000","last_clean":"2024-09-16T02:03:38.045513+0000","last_became_active":"2024-09-16T02:03:37.052409+0000","last_became_peered":"2024-09-16T02:03:37.052409+0000","last_unstale":"2024-09-16T02:03:38.045513+0000","last_undegraded":"2024-09-16T02:03:38.045513+0000","last_fullsized":"2024-09-16T02:03:38.045513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:52.667379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043821+0000","last_change":"2024-09-16T02:03:37.053645+0000","last_active":"2024-09-16T02:03:38.043821+0000","last_peered":"2024-09-16T02:03:38.043821+0000","last_clean":"2024-09-16T02:03:38.043821+0000","last_became_active":"2024-09-16T02:03:37.053340+0000","last_became_peered":"2024-09-16T02:03:37.053340+0000","last_unstale":"2024-09-16T02:03:38.043821+0000","last_undegraded":"2024-09-16T02:03:38.043821+0000","last_fullsized":"2024-09-16T02:03:38.043821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:59.827249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.044495+0000","last_change":"2024-09-16T02:03:34.744240+0000","last_active":"2024-09-16T02:03:38.044495+0000","last_peered":"2024-09-16T02:03:38.044495+0000","last_clean":"2024-09-16T02:03:38.044495+0000","last_became_active":"2024-09-16T02:03:34.743898+0000","last_became_peered":"2024-09-16T02:03:34.743898+0000","last_unstale":"2024-09-16T02:03:38.044495+0000","last_undegraded":"2024-09-16T02:03:38.044495+0000","last_fullsized":"2024-09-16T02:03:38.044495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:37.820529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035459+0000","last_change":"2024-09-16T02:03:37.055769+0000","last_active":"2024-09-16T02:03:38.035459+0000","last_peered":"2024-09-16T02:03:38.035459+0000","last_clean":"2024-09-16T02:03:38.035459+0000","last_became_active":"2024-09-16T02:03:37.054458+0000","last_became_peered":"2024-09-16T02:03:37.054458+0000","last_unstale":"2024-09-16T02:03:38.035459+0000","last_undegraded":"2024-09-16T02:03:38.035459+0000","last_fullsized":"2024-09-16T02:03:38.035459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:30.218712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.097979+0000","last_change":"2024-09-16T02:03:34.742514+0000","last_active":"2024-09-16T02:03:47.097979+0000","last_peered":"2024-09-16T02:03:47.097979+0000","last_clean":"2024-09-16T02:03:47.097979+0000","last_became_active":"2024-09-16T02:03:34.742123+0000","last_became_peered":"2024-09-16T02:03:34.742123+0000","last_unstale":"2024-09-16T02:03:47.097979+0000","last_undegraded":"2024-09-16T02:03:47.097979+0000","last_fullsized":"2024-09-16T02:03:47.097979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:10.956571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035343+0000","last_change":"2024-09-16T02:03:37.054370+0000","last_active":"2024-09-16T02:03:38.035343+0000","last_peered":"2024-09-16T02:03:38.035343+0000","last_clean":"2024-09-16T02:03:38.035343+0000","last_became_active":"2024-09-16T02:03:37.054051+0000","last_became_peered":"2024-09-16T02:03:37.054051+0000","last_unstale":"2024-09-16T02:03:38.035343+0000","last_undegraded":"2024-09-16T02:03:38.035343+0000","last_fullsized":"2024-09-16T02:03:38.035343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:21:42.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.518871+0000","last_change":"2024-09-16T02:03:34.739696+0000","last_active":"2024-09-16T02:03:38.518871+0000","last_peered":"2024-09-16T02:03:38.518871+0000","last_clean":"2024-09-16T02:03:38.518871+0000","last_became_active":"2024-09-16T02:03:34.738867+0000","last_became_peered":"2024-09-16T02:03:34.738867+0000","last_unstale":"2024-09-16T02:03:38.518871+0000","last_undegraded":"2024-09-16T02:03:38.518871+0000","last_fullsized":"2024-09-16T02:03:38.518871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:30.753970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048613+0000","last_change":"2024-09-16T02:03:37.058118+0000","last_active":"2024-09-16T02:03:38.048613+0000","last_peered":"2024-09-16T02:03:38.048613+0000","last_clean":"2024-09-16T02:03:38.048613+0000","last_became_active":"2024-09-16T02:03:37.057391+0000","last_became_peered":"2024-09-16T02:03:37.057391+0000","last_unstale":"2024-09-16T02:03:38.048613+0000","last_undegraded":"2024-09-16T02:03:38.048613+0000","last_fullsized":"2024-09-16T02:03:38.048613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:39.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.100296+0000","last_change":"2024-09-16T02:03:34.742184+0000","last_active":"2024-09-16T02:03:47.100296+0000","last_peered":"2024-09-16T02:03:47.100296+0000","last_clean":"2024-09-16T02:03:47.100296+0000","last_became_active":"2024-09-16T02:03:34.741838+0000","last_became_peered":"2024-09-16T02:03:34.741838+0000","last_unstale":"2024-09-16T02:03:47.100296+0000","last_undegraded":"2024-09-16T02:03:47.100296+0000","last_fullsized":"2024-09-16T02:03:47.100296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:57.338380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54508,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":115,"num_read_kb":114,"num_write":154,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":154477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678034,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":405846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1047832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037787,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":406082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266722,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":406002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495654,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1445888,"data_stored":998327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":405471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888112,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1458176,"data_stored":997618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051638,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":406301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280572,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":405782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476800,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":448072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607623,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":455376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":406742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":50112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:04:05.804 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:04:06.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:05 smithi136 ceph-mon[35323]: pgmap v295: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-09-16T02:04:06.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:05 smithi172 ceph-mon[35340]: pgmap v295: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-09-16T02:04:06.174 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:06.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.671+0000 7efccc000640 1 -- 172.21.15.103:0/1925548913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc4074590 msgr2=0x7efcc4074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:06.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.671+0000 7efccc000640 1 --2- 172.21.15.103:0/1925548913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc4074590 0x7efcc4074970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009900 tx=0x7efcb802a480 comp rx=0 tx=0).stop 2024-09-16T02:04:06.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.672+0000 7efccc000640 1 -- 172.21.15.103:0/1925548913 shutdown_connections 2024-09-16T02:04:06.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.672+0000 7efccc000640 1 --2- 172.21.15.103:0/1925548913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcc4112b90 0x7efcc4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.672+0000 7efccc000640 1 --2- 172.21.15.103:0/1925548913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc4074eb0 0x7efcc4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.672+0000 7efccc000640 1 --2- 172.21.15.103:0/1925548913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc4074590 0x7efcc4074970 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.672+0000 7efccc000640 1 -- 172.21.15.103:0/1925548913 >> 172.21.15.103:0/1925548913 conn(0x7efcc40fd8c0 msgr2=0x7efcc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:06.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.672+0000 7efccc000640 1 -- 172.21.15.103:0/1925548913 shutdown_connections 2024-09-16T02:04:06.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.672+0000 7efccc000640 1 -- 172.21.15.103:0/1925548913 wait complete. 2024-09-16T02:04:06.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.673+0000 7efccc000640 1 Processor -- start 2024-09-16T02:04:06.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.673+0000 7efccc000640 1 -- start start 2024-09-16T02:04:06.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.673+0000 7efccc000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcc4074590 0x7efcc41a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:06.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efccc000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc4074eb0 0x7efcc41a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efccc000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc4112b90 0x7efcc419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efccc000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41185a0 con 0x7efcc4074eb0 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efccc000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efcc4118420 con 0x7efcc4112b90 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efccc000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efcc4118720 con 0x7efcc4074590 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efcc8c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc4074eb0 0x7efcc41a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efcc8c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc4074eb0 0x7efcc41a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38486/0 (socket says 172.21.15.103:38486) 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efcc8c00640 1 -- 172.21.15.103:0/3054310279 learned_addr learned my addr 172.21.15.103:0/3054310279 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efcc9600640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcc4074590 0x7efcc41a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efcca000640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc4112b90 0x7efcc419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efcc8c00640 1 -- 172.21.15.103:0/3054310279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcc4074590 msgr2=0x7efcc41a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efcc8c00640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcc4074590 0x7efcc41a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efcc8c00640 1 -- 172.21.15.103:0/3054310279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc4112b90 msgr2=0x7efcc419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:06.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efcc8c00640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc4112b90 0x7efcc419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.674+0000 7efcc8c00640 1 -- 172.21.15.103:0/3054310279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc419efc0 con 0x7efcc4074eb0 2024-09-16T02:04:06.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.675+0000 7efcc8c00640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc4074eb0 0x7efcc41a4c10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efcb000e9f0 tx=0x7efcb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:06.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.675+0000 7efcc2000640 1 -- 172.21.15.103:0/3054310279 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb000cd80 con 0x7efcc4074eb0 2024-09-16T02:04:06.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.675+0000 7efcc2000640 1 -- 172.21.15.103:0/3054310279 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb0010910 con 0x7efcc4074eb0 2024-09-16T02:04:06.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.675+0000 7efccc000640 1 -- 172.21.15.103:0/3054310279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc419f2b0 con 0x7efcc4074eb0 2024-09-16T02:04:06.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.675+0000 7efcc2000640 1 -- 172.21.15.103:0/3054310279 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0018990 con 0x7efcc4074eb0 2024-09-16T02:04:06.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.675+0000 7efccc000640 1 -- 172.21.15.103:0/3054310279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc41ab5a0 con 0x7efcc4074eb0 2024-09-16T02:04:06.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.676+0000 7efccc000640 1 -- 172.21.15.103:0/3054310279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc4074970 con 0x7efcc4074eb0 2024-09-16T02:04:06.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.678+0000 7efcc2000640 1 -- 172.21.15.103:0/3054310279 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7efcb0010ac0 con 0x7efcc4074eb0 2024-09-16T02:04:06.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.680+0000 7efcc2000640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efcac0a3350 0x7efcac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:06.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.680+0000 7efcc2000640 1 -- 172.21.15.103:0/3054310279 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7efcb00cfc50 con 0x7efcc4074eb0 2024-09-16T02:04:06.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.680+0000 7efcc9600640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efcac0a3350 0x7efcac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:06.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.681+0000 7efcc9600640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efcac0a3350 0x7efcac0a5810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006010 tx=0x7efcb8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:06.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.683+0000 7efcc2000640 1 -- 172.21.15.103:0/3054310279 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcb009c270 con 0x7efcc4074eb0 2024-09-16T02:04:06.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.859+0000 7efccc000640 1 -- 172.21.15.103:0/3054310279 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efcc40ff6c0 con 0x7efcac0a3350 2024-09-16T02:04:06.860 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:06 smithi103 ceph-mon[30398]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.866+0000 7efcc2000640 1 -- 172.21.15.103:0/3054310279 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258701 (secure 0 0 0) 0x7efcc40ff6c0 con 0x7efcac0a3350 2024-09-16T02:04:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:06.870 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-16T02:04:06.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 -- 172.21.15.103:0/3054310279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efcac0a3350 msgr2=0x7efcac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efcac0a3350 0x7efcac0a5810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006010 tx=0x7efcb8000bc0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 -- 172.21.15.103:0/3054310279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc4074eb0 msgr2=0x7efcc41a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc4074eb0 0x7efcc41a4c10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efcb000e9f0 tx=0x7efcb000eeb0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 -- 172.21.15.103:0/3054310279 shutdown_connections 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efcac0a3350 0x7efcac0a5810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc4112b90 0x7efcc419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc4074eb0 0x7efcc41a4c10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 --2- 172.21.15.103:0/3054310279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcc4074590 0x7efcc41a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 -- 172.21.15.103:0/3054310279 >> 172.21.15.103:0/3054310279 conn(0x7efcc40fd8c0 msgr2=0x7efcc40fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 -- 172.21.15.103:0/3054310279 shutdown_connections 2024-09-16T02:04:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:06.870+0000 7efccc000640 1 -- 172.21.15.103:0/3054310279 wait complete. 2024-09-16T02:04:07.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:06 smithi136 ceph-mon[35323]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:07.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:06 smithi172 ceph-mon[35340]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:07.641 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-16T02:04:06.454453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645110,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":246,"num_read_kb":228,"num_write":344,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369460,"kb_used_data":11892,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572364,"statfs":{"total":1151940427776,"available":1151562100736,"internally_reserved":0,"allocated":12177408,"data_stored":6798498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42132,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":77,"num_write":33,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002987"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048779+0000","last_change":"2024-09-16T02:03:37.050873+0000","last_active":"2024-09-16T02:03:38.048779+0000","last_peered":"2024-09-16T02:03:38.048779+0000","last_clean":"2024-09-16T02:03:38.048779+0000","last_became_active":"2024-09-16T02:03:37.050202+0000","last_became_peered":"2024-09-16T02:03:37.050202+0000","last_unstale":"2024-09-16T02:03:38.048779+0000","last_undegraded":"2024-09-16T02:03:38.048779+0000","last_fullsized":"2024-09-16T02:03:38.048779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:12.971199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035219+0000","last_change":"2024-09-16T02:03:37.048034+0000","last_active":"2024-09-16T02:03:38.035219+0000","last_peered":"2024-09-16T02:03:38.035219+0000","last_clean":"2024-09-16T02:03:38.035219+0000","last_became_active":"2024-09-16T02:03:37.047216+0000","last_became_peered":"2024-09-16T02:03:37.047216+0000","last_unstale":"2024-09-16T02:03:38.035219+0000","last_undegraded":"2024-09-16T02:03:38.035219+0000","last_fullsized":"2024-09-16T02:03:38.035219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:48.580466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035331+0000","last_change":"2024-09-16T02:03:37.059287+0000","last_active":"2024-09-16T02:03:38.035331+0000","last_peered":"2024-09-16T02:03:38.035331+0000","last_clean":"2024-09-16T02:03:38.035331+0000","last_became_active":"2024-09-16T02:03:37.055270+0000","last_became_peered":"2024-09-16T02:03:37.055270+0000","last_unstale":"2024-09-16T02:03:38.035331+0000","last_undegraded":"2024-09-16T02:03:38.035331+0000","last_fullsized":"2024-09-16T02:03:38.035331+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:29.430514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046356+0000","last_change":"2024-09-16T02:03:37.052473+0000","last_active":"2024-09-16T02:03:38.046356+0000","last_peered":"2024-09-16T02:03:38.046356+0000","last_clean":"2024-09-16T02:03:38.046356+0000","last_became_active":"2024-09-16T02:03:37.052015+0000","last_became_peered":"2024-09-16T02:03:37.052015+0000","last_unstale":"2024-09-16T02:03:38.046356+0000","last_undegraded":"2024-09-16T02:03:38.046356+0000","last_fullsized":"2024-09-16T02:03:38.046356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:03.964132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.033262+0000","last_change":"2024-09-16T02:03:37.051516+0000","last_active":"2024-09-16T02:03:38.033262+0000","last_peered":"2024-09-16T02:03:38.033262+0000","last_clean":"2024-09-16T02:03:38.033262+0000","last_became_active":"2024-09-16T02:03:37.051201+0000","last_became_peered":"2024-09-16T02:03:37.051201+0000","last_unstale":"2024-09-16T02:03:38.033262+0000","last_undegraded":"2024-09-16T02:03:38.033262+0000","last_fullsized":"2024-09-16T02:03:38.033262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:45.327477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047424+0000","last_change":"2024-09-16T02:03:37.048912+0000","last_active":"2024-09-16T02:03:38.047424+0000","last_peered":"2024-09-16T02:03:38.047424+0000","last_clean":"2024-09-16T02:03:38.047424+0000","last_became_active":"2024-09-16T02:03:37.048577+0000","last_became_peered":"2024-09-16T02:03:37.048577+0000","last_unstale":"2024-09-16T02:03:38.047424+0000","last_undegraded":"2024-09-16T02:03:38.047424+0000","last_fullsized":"2024-09-16T02:03:38.047424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:25.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045070+0000","last_change":"2024-09-16T02:03:37.057427+0000","last_active":"2024-09-16T02:03:38.045070+0000","last_peered":"2024-09-16T02:03:38.045070+0000","last_clean":"2024-09-16T02:03:38.045070+0000","last_became_active":"2024-09-16T02:03:37.057111+0000","last_became_peered":"2024-09-16T02:03:37.057111+0000","last_unstale":"2024-09-16T02:03:38.045070+0000","last_undegraded":"2024-09-16T02:03:38.045070+0000","last_fullsized":"2024-09-16T02:03:38.045070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:44.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046741+0000","last_change":"2024-09-16T02:03:37.051085+0000","last_active":"2024-09-16T02:03:38.046741+0000","last_peered":"2024-09-16T02:03:38.046741+0000","last_clean":"2024-09-16T02:03:38.046741+0000","last_became_active":"2024-09-16T02:03:37.050790+0000","last_became_peered":"2024-09-16T02:03:37.050790+0000","last_unstale":"2024-09-16T02:03:38.046741+0000","last_undegraded":"2024-09-16T02:03:38.046741+0000","last_fullsized":"2024-09-16T02:03:38.046741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:18.620901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035545+0000","last_change":"2024-09-16T02:03:37.054258+0000","last_active":"2024-09-16T02:03:38.035545+0000","last_peered":"2024-09-16T02:03:38.035545+0000","last_clean":"2024-09-16T02:03:38.035545+0000","last_became_active":"2024-09-16T02:03:37.053939+0000","last_became_peered":"2024-09-16T02:03:37.053939+0000","last_unstale":"2024-09-16T02:03:38.035545+0000","last_undegraded":"2024-09-16T02:03:38.035545+0000","last_fullsized":"2024-09-16T02:03:38.035545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:11.076147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048385+0000","last_change":"2024-09-16T02:03:37.049828+0000","last_active":"2024-09-16T02:03:38.048385+0000","last_peered":"2024-09-16T02:03:38.048385+0000","last_clean":"2024-09-16T02:03:38.048385+0000","last_became_active":"2024-09-16T02:03:37.049577+0000","last_became_peered":"2024-09-16T02:03:37.049577+0000","last_unstale":"2024-09-16T02:03:38.048385+0000","last_undegraded":"2024-09-16T02:03:38.048385+0000","last_fullsized":"2024-09-16T02:03:38.048385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:37.176431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046251+0000","last_change":"2024-09-16T02:03:37.053268+0000","last_active":"2024-09-16T02:03:38.046251+0000","last_peered":"2024-09-16T02:03:38.046251+0000","last_clean":"2024-09-16T02:03:38.046251+0000","last_became_active":"2024-09-16T02:03:37.052933+0000","last_became_peered":"2024-09-16T02:03:37.052933+0000","last_unstale":"2024-09-16T02:03:38.046251+0000","last_undegraded":"2024-09-16T02:03:38.046251+0000","last_fullsized":"2024-09-16T02:03:38.046251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:32.003728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046516+0000","last_change":"2024-09-16T02:03:37.046339+0000","last_active":"2024-09-16T02:03:38.046516+0000","last_peered":"2024-09-16T02:03:38.046516+0000","last_clean":"2024-09-16T02:03:38.046516+0000","last_became_active":"2024-09-16T02:03:37.043262+0000","last_became_peered":"2024-09-16T02:03:37.043262+0000","last_unstale":"2024-09-16T02:03:38.046516+0000","last_undegraded":"2024-09-16T02:03:38.046516+0000","last_fullsized":"2024-09-16T02:03:38.046516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:58.614672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048257+0000","last_change":"2024-09-16T02:03:37.043856+0000","last_active":"2024-09-16T02:03:38.048257+0000","last_peered":"2024-09-16T02:03:38.048257+0000","last_clean":"2024-09-16T02:03:38.048257+0000","last_became_active":"2024-09-16T02:03:37.043546+0000","last_became_peered":"2024-09-16T02:03:37.043546+0000","last_unstale":"2024-09-16T02:03:38.048257+0000","last_undegraded":"2024-09-16T02:03:38.048257+0000","last_fullsized":"2024-09-16T02:03:38.048257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:57:07.213575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035119+0000","last_change":"2024-09-16T02:03:37.048844+0000","last_active":"2024-09-16T02:03:38.035119+0000","last_peered":"2024-09-16T02:03:38.035119+0000","last_clean":"2024-09-16T02:03:38.035119+0000","last_became_active":"2024-09-16T02:03:37.048522+0000","last_became_peered":"2024-09-16T02:03:37.048522+0000","last_unstale":"2024-09-16T02:03:38.035119+0000","last_undegraded":"2024-09-16T02:03:38.035119+0000","last_fullsized":"2024-09-16T02:03:38.035119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:50.759847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035028+0000","last_change":"2024-09-16T02:03:37.043236+0000","last_active":"2024-09-16T02:03:38.035028+0000","last_peered":"2024-09-16T02:03:38.035028+0000","last_clean":"2024-09-16T02:03:38.035028+0000","last_became_active":"2024-09-16T02:03:37.042967+0000","last_became_peered":"2024-09-16T02:03:37.042967+0000","last_unstale":"2024-09-16T02:03:38.035028+0000","last_undegraded":"2024-09-16T02:03:38.035028+0000","last_fullsized":"2024-09-16T02:03:38.035028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:02.739678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035121+0000","last_change":"2024-09-16T02:03:37.052972+0000","last_active":"2024-09-16T02:03:38.035121+0000","last_peered":"2024-09-16T02:03:38.035121+0000","last_clean":"2024-09-16T02:03:38.035121+0000","last_became_active":"2024-09-16T02:03:37.052665+0000","last_became_peered":"2024-09-16T02:03:37.052665+0000","last_unstale":"2024-09-16T02:03:38.035121+0000","last_undegraded":"2024-09-16T02:03:38.035121+0000","last_fullsized":"2024-09-16T02:03:38.035121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:59.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048818+0000","last_change":"2024-09-16T02:03:37.051203+0000","last_active":"2024-09-16T02:03:38.048818+0000","last_peered":"2024-09-16T02:03:38.048818+0000","last_clean":"2024-09-16T02:03:38.048818+0000","last_became_active":"2024-09-16T02:03:37.049828+0000","last_became_peered":"2024-09-16T02:03:37.049828+0000","last_unstale":"2024-09-16T02:03:38.048818+0000","last_undegraded":"2024-09-16T02:03:38.048818+0000","last_fullsized":"2024-09-16T02:03:38.048818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:47.566594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048163+0000","last_change":"2024-09-16T02:03:37.056903+0000","last_active":"2024-09-16T02:03:38.048163+0000","last_peered":"2024-09-16T02:03:38.048163+0000","last_clean":"2024-09-16T02:03:38.048163+0000","last_became_active":"2024-09-16T02:03:37.056579+0000","last_became_peered":"2024-09-16T02:03:37.056579+0000","last_unstale":"2024-09-16T02:03:38.048163+0000","last_undegraded":"2024-09-16T02:03:38.048163+0000","last_fullsized":"2024-09-16T02:03:38.048163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:23.652294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043469+0000","last_change":"2024-09-16T02:03:37.054413+0000","last_active":"2024-09-16T02:03:38.043469+0000","last_peered":"2024-09-16T02:03:38.043469+0000","last_clean":"2024-09-16T02:03:38.043469+0000","last_became_active":"2024-09-16T02:03:37.054040+0000","last_became_peered":"2024-09-16T02:03:37.054040+0000","last_unstale":"2024-09-16T02:03:38.043469+0000","last_undegraded":"2024-09-16T02:03:38.043469+0000","last_fullsized":"2024-09-16T02:03:38.043469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:04.517313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048372+0000","last_change":"2024-09-16T02:03:37.053454+0000","last_active":"2024-09-16T02:03:38.048372+0000","last_peered":"2024-09-16T02:03:38.048372+0000","last_clean":"2024-09-16T02:03:38.048372+0000","last_became_active":"2024-09-16T02:03:37.053123+0000","last_became_peered":"2024-09-16T02:03:37.053123+0000","last_unstale":"2024-09-16T02:03:38.048372+0000","last_undegraded":"2024-09-16T02:03:38.048372+0000","last_fullsized":"2024-09-16T02:03:38.048372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:52.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048111+0000","last_change":"2024-09-16T02:03:37.042772+0000","last_active":"2024-09-16T02:03:38.048111+0000","last_peered":"2024-09-16T02:03:38.048111+0000","last_clean":"2024-09-16T02:03:38.048111+0000","last_became_active":"2024-09-16T02:03:37.041832+0000","last_became_peered":"2024-09-16T02:03:37.041832+0000","last_unstale":"2024-09-16T02:03:38.048111+0000","last_undegraded":"2024-09-16T02:03:38.048111+0000","last_fullsized":"2024-09-16T02:03:38.048111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:54.436291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035219+0000","last_change":"2024-09-16T02:03:37.058484+0000","last_active":"2024-09-16T02:03:38.035219+0000","last_peered":"2024-09-16T02:03:38.035219+0000","last_clean":"2024-09-16T02:03:38.035219+0000","last_became_active":"2024-09-16T02:03:37.058047+0000","last_became_peered":"2024-09-16T02:03:37.058047+0000","last_unstale":"2024-09-16T02:03:38.035219+0000","last_undegraded":"2024-09-16T02:03:38.035219+0000","last_fullsized":"2024-09-16T02:03:38.035219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:23.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047461+0000","last_change":"2024-09-16T02:03:37.058576+0000","last_active":"2024-09-16T02:03:38.047461+0000","last_peered":"2024-09-16T02:03:38.047461+0000","last_clean":"2024-09-16T02:03:38.047461+0000","last_became_active":"2024-09-16T02:03:37.057624+0000","last_became_peered":"2024-09-16T02:03:37.057624+0000","last_unstale":"2024-09-16T02:03:38.047461+0000","last_undegraded":"2024-09-16T02:03:38.047461+0000","last_fullsized":"2024-09-16T02:03:38.047461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:21.929461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043131+0000","last_change":"2024-09-16T02:03:37.050889+0000","last_active":"2024-09-16T02:03:38.043131+0000","last_peered":"2024-09-16T02:03:38.043131+0000","last_clean":"2024-09-16T02:03:38.043131+0000","last_became_active":"2024-09-16T02:03:37.050535+0000","last_became_peered":"2024-09-16T02:03:37.050535+0000","last_unstale":"2024-09-16T02:03:38.043131+0000","last_undegraded":"2024-09-16T02:03:38.043131+0000","last_fullsized":"2024-09-16T02:03:38.043131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:46.955962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048934+0000","last_change":"2024-09-16T02:03:37.053865+0000","last_active":"2024-09-16T02:03:38.048934+0000","last_peered":"2024-09-16T02:03:38.048934+0000","last_clean":"2024-09-16T02:03:38.048934+0000","last_became_active":"2024-09-16T02:03:37.053563+0000","last_became_peered":"2024-09-16T02:03:37.053563+0000","last_unstale":"2024-09-16T02:03:38.048934+0000","last_undegraded":"2024-09-16T02:03:38.048934+0000","last_fullsized":"2024-09-16T02:03:38.048934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:24.109471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.042208+0000","last_change":"2024-09-16T02:03:37.054681+0000","last_active":"2024-09-16T02:03:38.042208+0000","last_peered":"2024-09-16T02:03:38.042208+0000","last_clean":"2024-09-16T02:03:38.042208+0000","last_became_active":"2024-09-16T02:03:37.054324+0000","last_became_peered":"2024-09-16T02:03:37.054324+0000","last_unstale":"2024-09-16T02:03:38.042208+0000","last_undegraded":"2024-09-16T02:03:38.042208+0000","last_fullsized":"2024-09-16T02:03:38.042208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:20.472520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048558+0000","last_change":"2024-09-16T02:03:37.047713+0000","last_active":"2024-09-16T02:03:38.048558+0000","last_peered":"2024-09-16T02:03:38.048558+0000","last_clean":"2024-09-16T02:03:38.048558+0000","last_became_active":"2024-09-16T02:03:37.047353+0000","last_became_peered":"2024-09-16T02:03:37.047353+0000","last_unstale":"2024-09-16T02:03:38.048558+0000","last_undegraded":"2024-09-16T02:03:38.048558+0000","last_fullsized":"2024-09-16T02:03:38.048558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:05.868909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047693+0000","last_change":"2024-09-16T02:03:37.038931+0000","last_active":"2024-09-16T02:03:38.047693+0000","last_peered":"2024-09-16T02:03:38.047693+0000","last_clean":"2024-09-16T02:03:38.047693+0000","last_became_active":"2024-09-16T02:03:37.037234+0000","last_became_peered":"2024-09-16T02:03:37.037234+0000","last_unstale":"2024-09-16T02:03:38.047693+0000","last_undegraded":"2024-09-16T02:03:38.047693+0000","last_fullsized":"2024-09-16T02:03:38.047693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:54.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035239+0000","last_change":"2024-09-16T02:03:37.051104+0000","last_active":"2024-09-16T02:03:38.035239+0000","last_peered":"2024-09-16T02:03:38.035239+0000","last_clean":"2024-09-16T02:03:38.035239+0000","last_became_active":"2024-09-16T02:03:37.048668+0000","last_became_peered":"2024-09-16T02:03:37.048668+0000","last_unstale":"2024-09-16T02:03:38.035239+0000","last_undegraded":"2024-09-16T02:03:38.035239+0000","last_fullsized":"2024-09-16T02:03:38.035239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:03.113818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.041767+0000","last_change":"2024-09-16T02:03:37.057688+0000","last_active":"2024-09-16T02:03:38.041767+0000","last_peered":"2024-09-16T02:03:38.041767+0000","last_clean":"2024-09-16T02:03:38.041767+0000","last_became_active":"2024-09-16T02:03:37.057347+0000","last_became_peered":"2024-09-16T02:03:37.057347+0000","last_unstale":"2024-09-16T02:03:38.041767+0000","last_undegraded":"2024-09-16T02:03:38.041767+0000","last_fullsized":"2024-09-16T02:03:38.041767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:30.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.042078+0000","last_change":"2024-09-16T02:03:37.054689+0000","last_active":"2024-09-16T02:03:38.042078+0000","last_peered":"2024-09-16T02:03:38.042078+0000","last_clean":"2024-09-16T02:03:38.042078+0000","last_became_active":"2024-09-16T02:03:37.054335+0000","last_became_peered":"2024-09-16T02:03:37.054335+0000","last_unstale":"2024-09-16T02:03:38.042078+0000","last_undegraded":"2024-09-16T02:03:38.042078+0000","last_fullsized":"2024-09-16T02:03:38.042078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:03.389582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048239+0000","last_change":"2024-09-16T02:03:37.043596+0000","last_active":"2024-09-16T02:03:38.048239+0000","last_peered":"2024-09-16T02:03:38.048239+0000","last_clean":"2024-09-16T02:03:38.048239+0000","last_became_active":"2024-09-16T02:03:37.043261+0000","last_became_peered":"2024-09-16T02:03:37.043261+0000","last_unstale":"2024-09-16T02:03:38.048239+0000","last_undegraded":"2024-09-16T02:03:38.048239+0000","last_fullsized":"2024-09-16T02:03:38.048239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:03.833126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.033285+0000","last_change":"2024-09-16T02:03:37.043018+0000","last_active":"2024-09-16T02:03:38.033285+0000","last_peered":"2024-09-16T02:03:38.033285+0000","last_clean":"2024-09-16T02:03:38.033285+0000","last_became_active":"2024-09-16T02:03:37.042437+0000","last_became_peered":"2024-09-16T02:03:37.042437+0000","last_unstale":"2024-09-16T02:03:38.033285+0000","last_undegraded":"2024-09-16T02:03:38.033285+0000","last_fullsized":"2024-09-16T02:03:38.033285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.193852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:04:01.123783+0000","last_change":"2024-09-16T02:03:34.741690+0000","last_active":"2024-09-16T02:04:01.123783+0000","last_peered":"2024-09-16T02:04:01.123783+0000","last_clean":"2024-09-16T02:04:01.123783+0000","last_became_active":"2024-09-16T02:03:34.739546+0000","last_became_peered":"2024-09-16T02:03:34.739546+0000","last_unstale":"2024-09-16T02:04:01.123783+0000","last_undegraded":"2024-09-16T02:04:01.123783+0000","last_fullsized":"2024-09-16T02:04:01.123783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:38.402329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043463+0000","last_change":"2024-09-16T02:03:37.052664+0000","last_active":"2024-09-16T02:03:38.043463+0000","last_peered":"2024-09-16T02:03:38.043463+0000","last_clean":"2024-09-16T02:03:38.043463+0000","last_became_active":"2024-09-16T02:03:37.051310+0000","last_became_peered":"2024-09-16T02:03:37.051310+0000","last_unstale":"2024-09-16T02:03:38.043463+0000","last_undegraded":"2024-09-16T02:03:38.043463+0000","last_fullsized":"2024-09-16T02:03:38.043463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:57.979053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.099870+0000","last_change":"2024-09-16T02:03:34.738020+0000","last_active":"2024-09-16T02:03:47.099870+0000","last_peered":"2024-09-16T02:03:47.099870+0000","last_clean":"2024-09-16T02:03:47.099870+0000","last_became_active":"2024-09-16T02:03:34.737644+0000","last_became_peered":"2024-09-16T02:03:34.737644+0000","last_unstale":"2024-09-16T02:03:47.099870+0000","last_undegraded":"2024-09-16T02:03:47.099870+0000","last_fullsized":"2024-09-16T02:03:47.099870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:56.249707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.100080+0000","last_change":"2024-09-16T02:03:34.745325+0000","last_active":"2024-09-16T02:03:47.100080+0000","last_peered":"2024-09-16T02:03:47.100080+0000","last_clean":"2024-09-16T02:03:47.100080+0000","last_became_active":"2024-09-16T02:03:34.744730+0000","last_became_peered":"2024-09-16T02:03:34.744730+0000","last_unstale":"2024-09-16T02:03:47.100080+0000","last_undegraded":"2024-09-16T02:03:47.100080+0000","last_fullsized":"2024-09-16T02:03:47.100080+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046876+0000","last_change":"2024-09-16T02:03:37.058973+0000","last_active":"2024-09-16T02:03:38.046876+0000","last_peered":"2024-09-16T02:03:38.046876+0000","last_clean":"2024-09-16T02:03:38.046876+0000","last_became_active":"2024-09-16T02:03:37.058685+0000","last_became_peered":"2024-09-16T02:03:37.058685+0000","last_unstale":"2024-09-16T02:03:38.046876+0000","last_undegraded":"2024-09-16T02:03:38.046876+0000","last_fullsized":"2024-09-16T02:03:38.046876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:46.030440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043011+0000","last_change":"2024-09-16T02:03:34.737842+0000","last_active":"2024-09-16T02:03:38.043011+0000","last_peered":"2024-09-16T02:03:38.043011+0000","last_clean":"2024-09-16T02:03:38.043011+0000","last_became_active":"2024-09-16T02:03:34.737391+0000","last_became_peered":"2024-09-16T02:03:34.737391+0000","last_unstale":"2024-09-16T02:03:38.043011+0000","last_undegraded":"2024-09-16T02:03:38.043011+0000","last_fullsized":"2024-09-16T02:03:38.043011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:39.214180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047367+0000","last_change":"2024-09-16T02:03:37.049705+0000","last_active":"2024-09-16T02:03:38.047367+0000","last_peered":"2024-09-16T02:03:38.047367+0000","last_clean":"2024-09-16T02:03:38.047367+0000","last_became_active":"2024-09-16T02:03:37.049392+0000","last_became_peered":"2024-09-16T02:03:37.049392+0000","last_unstale":"2024-09-16T02:03:38.047367+0000","last_undegraded":"2024-09-16T02:03:38.047367+0000","last_fullsized":"2024-09-16T02:03:38.047367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.932391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:04:01.123199+0000","last_change":"2024-09-16T02:03:34.742268+0000","last_active":"2024-09-16T02:04:01.123199+0000","last_peered":"2024-09-16T02:04:01.123199+0000","last_clean":"2024-09-16T02:04:01.123199+0000","last_became_active":"2024-09-16T02:03:34.741934+0000","last_became_peered":"2024-09-16T02:03:34.741934+0000","last_unstale":"2024-09-16T02:04:01.123199+0000","last_undegraded":"2024-09-16T02:04:01.123199+0000","last_fullsized":"2024-09-16T02:04:01.123199+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:30.301552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035453+0000","last_change":"2024-09-16T02:03:37.059119+0000","last_active":"2024-09-16T02:03:38.035453+0000","last_peered":"2024-09-16T02:03:38.035453+0000","last_clean":"2024-09-16T02:03:38.035453+0000","last_became_active":"2024-09-16T02:03:37.058738+0000","last_became_peered":"2024-09-16T02:03:37.058738+0000","last_unstale":"2024-09-16T02:03:38.035453+0000","last_undegraded":"2024-09-16T02:03:38.035453+0000","last_fullsized":"2024-09-16T02:03:38.035453+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.496053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.089084+0000","last_change":"2024-09-16T02:03:34.738291+0000","last_active":"2024-09-16T02:03:46.089084+0000","last_peered":"2024-09-16T02:03:46.089084+0000","last_clean":"2024-09-16T02:03:46.089084+0000","last_became_active":"2024-09-16T02:03:34.737436+0000","last_became_peered":"2024-09-16T02:03:34.737436+0000","last_unstale":"2024-09-16T02:03:46.089084+0000","last_undegraded":"2024-09-16T02:03:46.089084+0000","last_fullsized":"2024-09-16T02:03:46.089084+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:13.925417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035504+0000","last_change":"2024-09-16T02:03:37.050349+0000","last_active":"2024-09-16T02:03:38.035504+0000","last_peered":"2024-09-16T02:03:38.035504+0000","last_clean":"2024-09-16T02:03:38.035504+0000","last_became_active":"2024-09-16T02:03:37.049978+0000","last_became_peered":"2024-09-16T02:03:37.049978+0000","last_unstale":"2024-09-16T02:03:38.035504+0000","last_undegraded":"2024-09-16T02:03:38.035504+0000","last_fullsized":"2024-09-16T02:03:38.035504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:45.119683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.089329+0000","last_change":"2024-09-16T02:03:34.740246+0000","last_active":"2024-09-16T02:03:46.089329+0000","last_peered":"2024-09-16T02:03:46.089329+0000","last_clean":"2024-09-16T02:03:46.089329+0000","last_became_active":"2024-09-16T02:03:34.739908+0000","last_became_peered":"2024-09-16T02:03:34.739908+0000","last_unstale":"2024-09-16T02:03:46.089329+0000","last_undegraded":"2024-09-16T02:03:46.089329+0000","last_fullsized":"2024-09-16T02:03:46.089329+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:35.299733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035464+0000","last_change":"2024-09-16T02:03:37.056562+0000","last_active":"2024-09-16T02:03:38.035464+0000","last_peered":"2024-09-16T02:03:38.035464+0000","last_clean":"2024-09-16T02:03:38.035464+0000","last_became_active":"2024-09-16T02:03:37.056226+0000","last_became_peered":"2024-09-16T02:03:37.056226+0000","last_unstale":"2024-09-16T02:03:38.035464+0000","last_undegraded":"2024-09-16T02:03:38.035464+0000","last_fullsized":"2024-09-16T02:03:38.035464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.741538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'25","reported_seq":80,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:04:02.126557+0000","last_change":"2024-09-16T02:03:34.736403+0000","last_active":"2024-09-16T02:04:02.126557+0000","last_peered":"2024-09-16T02:04:02.126557+0000","last_clean":"2024-09-16T02:04:02.126557+0000","last_became_active":"2024-09-16T02:03:34.736074+0000","last_became_peered":"2024-09-16T02:03:34.736074+0000","last_unstale":"2024-09-16T02:04:02.126557+0000","last_undegraded":"2024-09-16T02:04:02.126557+0000","last_fullsized":"2024-09-16T02:04:02.126557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:34.769862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50112,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":66,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.033652+0000","last_change":"2024-09-16T02:03:37.046803+0000","last_active":"2024-09-16T02:03:38.033652+0000","last_peered":"2024-09-16T02:03:38.033652+0000","last_clean":"2024-09-16T02:03:38.033652+0000","last_became_active":"2024-09-16T02:03:37.046459+0000","last_became_peered":"2024-09-16T02:03:37.046459+0000","last_unstale":"2024-09-16T02:03:38.033652+0000","last_undegraded":"2024-09-16T02:03:38.033652+0000","last_fullsized":"2024-09-16T02:03:38.033652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:55:38.460795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.099275+0000","last_change":"2024-09-16T02:03:34.739016+0000","last_active":"2024-09-16T02:03:47.099275+0000","last_peered":"2024-09-16T02:03:47.099275+0000","last_clean":"2024-09-16T02:03:47.099275+0000","last_became_active":"2024-09-16T02:03:34.738583+0000","last_became_peered":"2024-09-16T02:03:34.738583+0000","last_unstale":"2024-09-16T02:03:47.099275+0000","last_undegraded":"2024-09-16T02:03:47.099275+0000","last_fullsized":"2024-09-16T02:03:47.099275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:08.255832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046158+0000","last_change":"2024-09-16T02:03:37.053353+0000","last_active":"2024-09-16T02:03:38.046158+0000","last_peered":"2024-09-16T02:03:38.046158+0000","last_clean":"2024-09-16T02:03:38.046158+0000","last_became_active":"2024-09-16T02:03:37.051140+0000","last_became_peered":"2024-09-16T02:03:37.051140+0000","last_unstale":"2024-09-16T02:03:38.046158+0000","last_undegraded":"2024-09-16T02:03:38.046158+0000","last_fullsized":"2024-09-16T02:03:38.046158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:34.636338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.089256+0000","last_change":"2024-09-16T02:03:34.742111+0000","last_active":"2024-09-16T02:03:46.089256+0000","last_peered":"2024-09-16T02:03:46.089256+0000","last_clean":"2024-09-16T02:03:46.089256+0000","last_became_active":"2024-09-16T02:03:34.741816+0000","last_became_peered":"2024-09-16T02:03:34.741816+0000","last_unstale":"2024-09-16T02:03:46.089256+0000","last_undegraded":"2024-09-16T02:03:46.089256+0000","last_fullsized":"2024-09-16T02:03:46.089256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:36.861995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046067+0000","last_change":"2024-09-16T02:03:37.047152+0000","last_active":"2024-09-16T02:03:38.046067+0000","last_peered":"2024-09-16T02:03:38.046067+0000","last_clean":"2024-09-16T02:03:38.046067+0000","last_became_active":"2024-09-16T02:03:37.046675+0000","last_became_peered":"2024-09-16T02:03:37.046675+0000","last_unstale":"2024-09-16T02:03:38.046067+0000","last_undegraded":"2024-09-16T02:03:38.046067+0000","last_fullsized":"2024-09-16T02:03:38.046067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:58.236519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:04:01.123220+0000","last_change":"2024-09-16T02:03:34.738487+0000","last_active":"2024-09-16T02:04:01.123220+0000","last_peered":"2024-09-16T02:04:01.123220+0000","last_clean":"2024-09-16T02:04:01.123220+0000","last_became_active":"2024-09-16T02:03:34.737061+0000","last_became_peered":"2024-09-16T02:03:34.737061+0000","last_unstale":"2024-09-16T02:04:01.123220+0000","last_undegraded":"2024-09-16T02:04:01.123220+0000","last_fullsized":"2024-09-16T02:04:01.123220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:24.231640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045125+0000","last_change":"2024-09-16T02:03:37.044260+0000","last_active":"2024-09-16T02:03:38.045125+0000","last_peered":"2024-09-16T02:03:38.045125+0000","last_clean":"2024-09-16T02:03:38.045125+0000","last_became_active":"2024-09-16T02:03:37.043932+0000","last_became_peered":"2024-09-16T02:03:37.043932+0000","last_unstale":"2024-09-16T02:03:38.045125+0000","last_undegraded":"2024-09-16T02:03:38.045125+0000","last_fullsized":"2024-09-16T02:03:38.045125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:55.718155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.100668+0000","last_change":"2024-09-16T02:03:34.742451+0000","last_active":"2024-09-16T02:03:47.100668+0000","last_peered":"2024-09-16T02:03:47.100668+0000","last_clean":"2024-09-16T02:03:47.100668+0000","last_became_active":"2024-09-16T02:03:34.742089+0000","last_became_peered":"2024-09-16T02:03:34.742089+0000","last_unstale":"2024-09-16T02:03:47.100668+0000","last_undegraded":"2024-09-16T02:03:47.100668+0000","last_fullsized":"2024-09-16T02:03:47.100668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:20.426059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043675+0000","last_change":"2024-09-16T02:03:37.053040+0000","last_active":"2024-09-16T02:03:38.043675+0000","last_peered":"2024-09-16T02:03:38.043675+0000","last_clean":"2024-09-16T02:03:38.043675+0000","last_became_active":"2024-09-16T02:03:37.052185+0000","last_became_peered":"2024-09-16T02:03:37.052185+0000","last_unstale":"2024-09-16T02:03:38.043675+0000","last_undegraded":"2024-09-16T02:03:38.043675+0000","last_fullsized":"2024-09-16T02:03:38.043675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:27.676678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:48.094783+0000","last_change":"2024-09-16T02:03:34.740234+0000","last_active":"2024-09-16T02:03:48.094783+0000","last_peered":"2024-09-16T02:03:48.094783+0000","last_clean":"2024-09-16T02:03:48.094783+0000","last_became_active":"2024-09-16T02:03:34.739905+0000","last_became_peered":"2024-09-16T02:03:34.739905+0000","last_unstale":"2024-09-16T02:03:48.094783+0000","last_undegraded":"2024-09-16T02:03:48.094783+0000","last_fullsized":"2024-09-16T02:03:48.094783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:19:02.347369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048636+0000","last_change":"2024-09-16T02:03:37.049420+0000","last_active":"2024-09-16T02:03:38.048636+0000","last_peered":"2024-09-16T02:03:38.048636+0000","last_clean":"2024-09-16T02:03:38.048636+0000","last_became_active":"2024-09-16T02:03:37.047671+0000","last_became_peered":"2024-09-16T02:03:37.047671+0000","last_unstale":"2024-09-16T02:03:38.048636+0000","last_undegraded":"2024-09-16T02:03:38.048636+0000","last_fullsized":"2024-09-16T02:03:38.048636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:23.023791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.050781+0000","last_change":"2024-09-16T02:03:34.738516+0000","last_active":"2024-09-16T02:03:38.050781+0000","last_peered":"2024-09-16T02:03:38.050781+0000","last_clean":"2024-09-16T02:03:38.050781+0000","last_became_active":"2024-09-16T02:03:34.737701+0000","last_became_peered":"2024-09-16T02:03:34.737701+0000","last_unstale":"2024-09-16T02:03:38.050781+0000","last_undegraded":"2024-09-16T02:03:38.050781+0000","last_fullsized":"2024-09-16T02:03:38.050781+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:42.101865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045953+0000","last_change":"2024-09-16T02:03:37.046820+0000","last_active":"2024-09-16T02:03:38.045953+0000","last_peered":"2024-09-16T02:03:38.045953+0000","last_clean":"2024-09-16T02:03:38.045953+0000","last_became_active":"2024-09-16T02:03:37.046082+0000","last_became_peered":"2024-09-16T02:03:37.046082+0000","last_unstale":"2024-09-16T02:03:38.045953+0000","last_undegraded":"2024-09-16T02:03:38.045953+0000","last_fullsized":"2024-09-16T02:03:38.045953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:21.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046593+0000","last_change":"2024-09-16T02:03:34.741908+0000","last_active":"2024-09-16T02:03:38.046593+0000","last_peered":"2024-09-16T02:03:38.046593+0000","last_clean":"2024-09-16T02:03:38.046593+0000","last_became_active":"2024-09-16T02:03:34.739826+0000","last_became_peered":"2024-09-16T02:03:34.739826+0000","last_unstale":"2024-09-16T02:03:38.046593+0000","last_undegraded":"2024-09-16T02:03:38.046593+0000","last_fullsized":"2024-09-16T02:03:38.046593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:04.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:40.502622+0000","last_change":"2024-09-16T02:03:37.056459+0000","last_active":"2024-09-16T02:03:40.502622+0000","last_peered":"2024-09-16T02:03:40.502622+0000","last_clean":"2024-09-16T02:03:40.502622+0000","last_became_active":"2024-09-16T02:03:37.055902+0000","last_became_peered":"2024-09-16T02:03:37.055902+0000","last_unstale":"2024-09-16T02:03:40.502622+0000","last_undegraded":"2024-09-16T02:03:40.502622+0000","last_fullsized":"2024-09-16T02:03:40.502622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:48:30.815654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.100170+0000","last_change":"2024-09-16T02:03:34.741207+0000","last_active":"2024-09-16T02:03:47.100170+0000","last_peered":"2024-09-16T02:03:47.100170+0000","last_clean":"2024-09-16T02:03:47.100170+0000","last_became_active":"2024-09-16T02:03:34.740523+0000","last_became_peered":"2024-09-16T02:03:34.740523+0000","last_unstale":"2024-09-16T02:03:47.100170+0000","last_undegraded":"2024-09-16T02:03:47.100170+0000","last_fullsized":"2024-09-16T02:03:47.100170+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:36.199332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.046291+0000","last_change":"2024-09-16T02:03:37.052792+0000","last_active":"2024-09-16T02:03:38.046291+0000","last_peered":"2024-09-16T02:03:38.046291+0000","last_clean":"2024-09-16T02:03:38.046291+0000","last_became_active":"2024-09-16T02:03:37.052301+0000","last_became_peered":"2024-09-16T02:03:37.052301+0000","last_unstale":"2024-09-16T02:03:38.046291+0000","last_undegraded":"2024-09-16T02:03:38.046291+0000","last_fullsized":"2024-09-16T02:03:38.046291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:55.197481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":329,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045809+0000","last_change":"2024-09-16T02:02:21.095893+0000","last_active":"2024-09-16T02:03:38.045809+0000","last_peered":"2024-09-16T02:03:38.045809+0000","last_clean":"2024-09-16T02:03:38.045809+0000","last_became_active":"2024-09-16T02:02:21.095395+0000","last_became_peered":"2024-09-16T02:02:21.095395+0000","last_unstale":"2024-09-16T02:03:38.045809+0000","last_undegraded":"2024-09-16T02:03:38.045809+0000","last_fullsized":"2024-09-16T02:03:38.045809+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_clean_scrub_stamp":"2024-09-16T01:58:50.954254+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:40.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:48.100136+0000","last_change":"2024-09-16T02:03:34.746154+0000","last_active":"2024-09-16T02:03:48.100136+0000","last_peered":"2024-09-16T02:03:48.100136+0000","last_clean":"2024-09-16T02:03:48.100136+0000","last_became_active":"2024-09-16T02:03:34.745646+0000","last_became_peered":"2024-09-16T02:03:34.745646+0000","last_unstale":"2024-09-16T02:03:48.100136+0000","last_undegraded":"2024-09-16T02:03:48.100136+0000","last_fullsized":"2024-09-16T02:03:48.100136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:06.790000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047338+0000","last_change":"2024-09-16T02:03:37.056638+0000","last_active":"2024-09-16T02:03:38.047338+0000","last_peered":"2024-09-16T02:03:38.047338+0000","last_clean":"2024-09-16T02:03:38.047338+0000","last_became_active":"2024-09-16T02:03:37.056043+0000","last_became_peered":"2024-09-16T02:03:37.056043+0000","last_unstale":"2024-09-16T02:03:38.047338+0000","last_undegraded":"2024-09-16T02:03:38.047338+0000","last_fullsized":"2024-09-16T02:03:38.047338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:35:48.353380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.098205+0000","last_change":"2024-09-16T02:03:34.741453+0000","last_active":"2024-09-16T02:03:47.098205+0000","last_peered":"2024-09-16T02:03:47.098205+0000","last_clean":"2024-09-16T02:03:47.098205+0000","last_became_active":"2024-09-16T02:03:34.741128+0000","last_became_peered":"2024-09-16T02:03:34.741128+0000","last_unstale":"2024-09-16T02:03:47.098205+0000","last_undegraded":"2024-09-16T02:03:47.098205+0000","last_fullsized":"2024-09-16T02:03:47.098205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:41.544575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045992+0000","last_change":"2024-09-16T02:03:37.050895+0000","last_active":"2024-09-16T02:03:38.045992+0000","last_peered":"2024-09-16T02:03:38.045992+0000","last_clean":"2024-09-16T02:03:38.045992+0000","last_became_active":"2024-09-16T02:03:37.050559+0000","last_became_peered":"2024-09-16T02:03:37.050559+0000","last_unstale":"2024-09-16T02:03:38.045992+0000","last_undegraded":"2024-09-16T02:03:38.045992+0000","last_fullsized":"2024-09-16T02:03:38.045992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:20.067954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:04:04.127254+0000","last_change":"2024-09-16T02:03:34.740286+0000","last_active":"2024-09-16T02:04:04.127254+0000","last_peered":"2024-09-16T02:04:04.127254+0000","last_clean":"2024-09-16T02:04:04.127254+0000","last_became_active":"2024-09-16T02:03:34.739892+0000","last_became_peered":"2024-09-16T02:03:34.739892+0000","last_unstale":"2024-09-16T02:04:04.127254+0000","last_undegraded":"2024-09-16T02:04:04.127254+0000","last_fullsized":"2024-09-16T02:04:04.127254+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:59:14.809991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.042775+0000","last_change":"2024-09-16T02:03:37.042207+0000","last_active":"2024-09-16T02:03:38.042775+0000","last_peered":"2024-09-16T02:03:38.042775+0000","last_clean":"2024-09-16T02:03:38.042775+0000","last_became_active":"2024-09-16T02:03:37.041283+0000","last_became_peered":"2024-09-16T02:03:37.041283+0000","last_unstale":"2024-09-16T02:03:38.042775+0000","last_undegraded":"2024-09-16T02:03:38.042775+0000","last_fullsized":"2024-09-16T02:03:38.042775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:45.569721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045742+0000","last_change":"2024-09-16T02:03:34.737834+0000","last_active":"2024-09-16T02:03:38.045742+0000","last_peered":"2024-09-16T02:03:38.045742+0000","last_clean":"2024-09-16T02:03:38.045742+0000","last_became_active":"2024-09-16T02:03:34.736706+0000","last_became_peered":"2024-09-16T02:03:34.736706+0000","last_unstale":"2024-09-16T02:03:38.045742+0000","last_undegraded":"2024-09-16T02:03:38.045742+0000","last_fullsized":"2024-09-16T02:03:38.045742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:27.037351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047783+0000","last_change":"2024-09-16T02:03:37.060888+0000","last_active":"2024-09-16T02:03:38.047783+0000","last_peered":"2024-09-16T02:03:38.047783+0000","last_clean":"2024-09-16T02:03:38.047783+0000","last_became_active":"2024-09-16T02:03:37.060458+0000","last_became_peered":"2024-09-16T02:03:37.060458+0000","last_unstale":"2024-09-16T02:03:38.047783+0000","last_undegraded":"2024-09-16T02:03:38.047783+0000","last_fullsized":"2024-09-16T02:03:38.047783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:37.425376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047887+0000","last_change":"2024-09-16T02:03:34.742298+0000","last_active":"2024-09-16T02:03:38.047887+0000","last_peered":"2024-09-16T02:03:38.047887+0000","last_clean":"2024-09-16T02:03:38.047887+0000","last_became_active":"2024-09-16T02:03:34.741960+0000","last_became_peered":"2024-09-16T02:03:34.741960+0000","last_unstale":"2024-09-16T02:03:38.047887+0000","last_undegraded":"2024-09-16T02:03:38.047887+0000","last_fullsized":"2024-09-16T02:03:38.047887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:12.832207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047039+0000","last_change":"2024-09-16T02:03:37.060553+0000","last_active":"2024-09-16T02:03:38.047039+0000","last_peered":"2024-09-16T02:03:38.047039+0000","last_clean":"2024-09-16T02:03:38.047039+0000","last_became_active":"2024-09-16T02:03:37.059961+0000","last_became_peered":"2024-09-16T02:03:37.059961+0000","last_unstale":"2024-09-16T02:03:38.047039+0000","last_undegraded":"2024-09-16T02:03:38.047039+0000","last_fullsized":"2024-09-16T02:03:38.047039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:22.904190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.087975+0000","last_change":"2024-09-16T02:03:34.747828+0000","last_active":"2024-09-16T02:03:46.087975+0000","last_peered":"2024-09-16T02:03:46.087975+0000","last_clean":"2024-09-16T02:03:46.087975+0000","last_became_active":"2024-09-16T02:03:34.747548+0000","last_became_peered":"2024-09-16T02:03:34.747548+0000","last_unstale":"2024-09-16T02:03:46.087975+0000","last_undegraded":"2024-09-16T02:03:46.087975+0000","last_fullsized":"2024-09-16T02:03:46.087975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:49.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047552+0000","last_change":"2024-09-16T02:03:37.051477+0000","last_active":"2024-09-16T02:03:38.047552+0000","last_peered":"2024-09-16T02:03:38.047552+0000","last_clean":"2024-09-16T02:03:38.047552+0000","last_became_active":"2024-09-16T02:03:37.048063+0000","last_became_peered":"2024-09-16T02:03:37.048063+0000","last_unstale":"2024-09-16T02:03:38.047552+0000","last_undegraded":"2024-09-16T02:03:38.047552+0000","last_fullsized":"2024-09-16T02:03:38.047552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:15.973972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:04:03.040892+0000","last_change":"2024-09-16T02:03:34.742935+0000","last_active":"2024-09-16T02:04:03.040892+0000","last_peered":"2024-09-16T02:04:03.040892+0000","last_clean":"2024-09-16T02:04:03.040892+0000","last_became_active":"2024-09-16T02:03:34.742611+0000","last_became_peered":"2024-09-16T02:03:34.742611+0000","last_unstale":"2024-09-16T02:04:03.040892+0000","last_undegraded":"2024-09-16T02:04:03.040892+0000","last_fullsized":"2024-09-16T02:04:03.040892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:36.695220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.049294+0000","last_change":"2024-09-16T02:03:37.054926+0000","last_active":"2024-09-16T02:03:38.049294+0000","last_peered":"2024-09-16T02:03:38.049294+0000","last_clean":"2024-09-16T02:03:38.049294+0000","last_became_active":"2024-09-16T02:03:37.054483+0000","last_became_peered":"2024-09-16T02:03:37.054483+0000","last_unstale":"2024-09-16T02:03:38.049294+0000","last_undegraded":"2024-09-16T02:03:38.049294+0000","last_fullsized":"2024-09-16T02:03:38.049294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.090767+0000","last_change":"2024-09-16T02:03:34.739611+0000","last_active":"2024-09-16T02:03:46.090767+0000","last_peered":"2024-09-16T02:03:46.090767+0000","last_clean":"2024-09-16T02:03:46.090767+0000","last_became_active":"2024-09-16T02:03:34.738090+0000","last_became_peered":"2024-09-16T02:03:34.738090+0000","last_unstale":"2024-09-16T02:03:46.090767+0000","last_undegraded":"2024-09-16T02:03:46.090767+0000","last_fullsized":"2024-09-16T02:03:46.090767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:47.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035368+0000","last_change":"2024-09-16T02:03:37.053020+0000","last_active":"2024-09-16T02:03:38.035368+0000","last_peered":"2024-09-16T02:03:38.035368+0000","last_clean":"2024-09-16T02:03:38.035368+0000","last_became_active":"2024-09-16T02:03:37.052365+0000","last_became_peered":"2024-09-16T02:03:37.052365+0000","last_unstale":"2024-09-16T02:03:38.035368+0000","last_undegraded":"2024-09-16T02:03:38.035368+0000","last_fullsized":"2024-09-16T02:03:38.035368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:54.965028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.089414+0000","last_change":"2024-09-16T02:03:34.740726+0000","last_active":"2024-09-16T02:03:46.089414+0000","last_peered":"2024-09-16T02:03:46.089414+0000","last_clean":"2024-09-16T02:03:46.089414+0000","last_became_active":"2024-09-16T02:03:34.740402+0000","last_became_peered":"2024-09-16T02:03:34.740402+0000","last_unstale":"2024-09-16T02:03:46.089414+0000","last_undegraded":"2024-09-16T02:03:46.089414+0000","last_fullsized":"2024-09-16T02:03:46.089414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:05.761646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.033218+0000","last_change":"2024-09-16T02:03:37.044862+0000","last_active":"2024-09-16T02:03:38.033218+0000","last_peered":"2024-09-16T02:03:38.033218+0000","last_clean":"2024-09-16T02:03:38.033218+0000","last_became_active":"2024-09-16T02:03:37.044406+0000","last_became_peered":"2024-09-16T02:03:37.044406+0000","last_unstale":"2024-09-16T02:03:38.033218+0000","last_undegraded":"2024-09-16T02:03:38.033218+0000","last_fullsized":"2024-09-16T02:03:38.033218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:03.326380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.092552+0000","last_change":"2024-09-16T02:03:34.745505+0000","last_active":"2024-09-16T02:03:46.092552+0000","last_peered":"2024-09-16T02:03:46.092552+0000","last_clean":"2024-09-16T02:03:46.092552+0000","last_became_active":"2024-09-16T02:03:34.745151+0000","last_became_peered":"2024-09-16T02:03:34.745151+0000","last_unstale":"2024-09-16T02:03:46.092552+0000","last_undegraded":"2024-09-16T02:03:46.092552+0000","last_fullsized":"2024-09-16T02:03:46.092552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:57.401176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048300+0000","last_change":"2024-09-16T02:03:37.046692+0000","last_active":"2024-09-16T02:03:38.048300+0000","last_peered":"2024-09-16T02:03:38.048300+0000","last_clean":"2024-09-16T02:03:38.048300+0000","last_became_active":"2024-09-16T02:03:37.046302+0000","last_became_peered":"2024-09-16T02:03:37.046302+0000","last_unstale":"2024-09-16T02:03:38.048300+0000","last_undegraded":"2024-09-16T02:03:38.048300+0000","last_fullsized":"2024-09-16T02:03:38.048300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:47.727431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.041918+0000","last_change":"2024-09-16T02:03:34.739847+0000","last_active":"2024-09-16T02:03:38.041918+0000","last_peered":"2024-09-16T02:03:38.041918+0000","last_clean":"2024-09-16T02:03:38.041918+0000","last_became_active":"2024-09-16T02:03:34.739514+0000","last_became_peered":"2024-09-16T02:03:34.739514+0000","last_unstale":"2024-09-16T02:03:38.041918+0000","last_undegraded":"2024-09-16T02:03:38.041918+0000","last_fullsized":"2024-09-16T02:03:38.041918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:44.617127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.047337+0000","last_change":"2024-09-16T02:03:37.054910+0000","last_active":"2024-09-16T02:03:38.047337+0000","last_peered":"2024-09-16T02:03:38.047337+0000","last_clean":"2024-09-16T02:03:38.047337+0000","last_became_active":"2024-09-16T02:03:37.053852+0000","last_became_peered":"2024-09-16T02:03:37.053852+0000","last_unstale":"2024-09-16T02:03:38.047337+0000","last_undegraded":"2024-09-16T02:03:38.047337+0000","last_fullsized":"2024-09-16T02:03:38.047337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:29.042154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:46.089808+0000","last_change":"2024-09-16T02:03:34.743027+0000","last_active":"2024-09-16T02:03:46.089808+0000","last_peered":"2024-09-16T02:03:46.089808+0000","last_clean":"2024-09-16T02:03:46.089808+0000","last_became_active":"2024-09-16T02:03:34.742658+0000","last_became_peered":"2024-09-16T02:03:34.742658+0000","last_unstale":"2024-09-16T02:03:46.089808+0000","last_undegraded":"2024-09-16T02:03:46.089808+0000","last_fullsized":"2024-09-16T02:03:46.089808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:36.779568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.045513+0000","last_change":"2024-09-16T02:03:37.052783+0000","last_active":"2024-09-16T02:03:38.045513+0000","last_peered":"2024-09-16T02:03:38.045513+0000","last_clean":"2024-09-16T02:03:38.045513+0000","last_became_active":"2024-09-16T02:03:37.052409+0000","last_became_peered":"2024-09-16T02:03:37.052409+0000","last_unstale":"2024-09-16T02:03:38.045513+0000","last_undegraded":"2024-09-16T02:03:38.045513+0000","last_fullsized":"2024-09-16T02:03:38.045513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:52.667379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.043821+0000","last_change":"2024-09-16T02:03:37.053645+0000","last_active":"2024-09-16T02:03:38.043821+0000","last_peered":"2024-09-16T02:03:38.043821+0000","last_clean":"2024-09-16T02:03:38.043821+0000","last_became_active":"2024-09-16T02:03:37.053340+0000","last_became_peered":"2024-09-16T02:03:37.053340+0000","last_unstale":"2024-09-16T02:03:38.043821+0000","last_undegraded":"2024-09-16T02:03:38.043821+0000","last_fullsized":"2024-09-16T02:03:38.043821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:59.827249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.044495+0000","last_change":"2024-09-16T02:03:34.744240+0000","last_active":"2024-09-16T02:03:38.044495+0000","last_peered":"2024-09-16T02:03:38.044495+0000","last_clean":"2024-09-16T02:03:38.044495+0000","last_became_active":"2024-09-16T02:03:34.743898+0000","last_became_peered":"2024-09-16T02:03:34.743898+0000","last_unstale":"2024-09-16T02:03:38.044495+0000","last_undegraded":"2024-09-16T02:03:38.044495+0000","last_fullsized":"2024-09-16T02:03:38.044495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:37.820529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035459+0000","last_change":"2024-09-16T02:03:37.055769+0000","last_active":"2024-09-16T02:03:38.035459+0000","last_peered":"2024-09-16T02:03:38.035459+0000","last_clean":"2024-09-16T02:03:38.035459+0000","last_became_active":"2024-09-16T02:03:37.054458+0000","last_became_peered":"2024-09-16T02:03:37.054458+0000","last_unstale":"2024-09-16T02:03:38.035459+0000","last_undegraded":"2024-09-16T02:03:38.035459+0000","last_fullsized":"2024-09-16T02:03:38.035459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:30.218712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.097979+0000","last_change":"2024-09-16T02:03:34.742514+0000","last_active":"2024-09-16T02:03:47.097979+0000","last_peered":"2024-09-16T02:03:47.097979+0000","last_clean":"2024-09-16T02:03:47.097979+0000","last_became_active":"2024-09-16T02:03:34.742123+0000","last_became_peered":"2024-09-16T02:03:34.742123+0000","last_unstale":"2024-09-16T02:03:47.097979+0000","last_undegraded":"2024-09-16T02:03:47.097979+0000","last_fullsized":"2024-09-16T02:03:47.097979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:10.956571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.035343+0000","last_change":"2024-09-16T02:03:37.054370+0000","last_active":"2024-09-16T02:03:38.035343+0000","last_peered":"2024-09-16T02:03:38.035343+0000","last_clean":"2024-09-16T02:03:38.035343+0000","last_became_active":"2024-09-16T02:03:37.054051+0000","last_became_peered":"2024-09-16T02:03:37.054051+0000","last_unstale":"2024-09-16T02:03:38.035343+0000","last_undegraded":"2024-09-16T02:03:38.035343+0000","last_fullsized":"2024-09-16T02:03:38.035343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:21:42.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.518871+0000","last_change":"2024-09-16T02:03:34.739696+0000","last_active":"2024-09-16T02:03:38.518871+0000","last_peered":"2024-09-16T02:03:38.518871+0000","last_clean":"2024-09-16T02:03:38.518871+0000","last_became_active":"2024-09-16T02:03:34.738867+0000","last_became_peered":"2024-09-16T02:03:34.738867+0000","last_unstale":"2024-09-16T02:03:38.518871+0000","last_undegraded":"2024-09-16T02:03:38.518871+0000","last_fullsized":"2024-09-16T02:03:38.518871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:30.753970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:38.048613+0000","last_change":"2024-09-16T02:03:37.058118+0000","last_active":"2024-09-16T02:03:38.048613+0000","last_peered":"2024-09-16T02:03:38.048613+0000","last_clean":"2024-09-16T02:03:38.048613+0000","last_became_active":"2024-09-16T02:03:37.057391+0000","last_became_peered":"2024-09-16T02:03:37.057391+0000","last_unstale":"2024-09-16T02:03:38.048613+0000","last_undegraded":"2024-09-16T02:03:38.048613+0000","last_fullsized":"2024-09-16T02:03:38.048613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:39.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T02:03:47.100296+0000","last_change":"2024-09-16T02:03:34.742184+0000","last_active":"2024-09-16T02:03:47.100296+0000","last_peered":"2024-09-16T02:03:47.100296+0000","last_clean":"2024-09-16T02:03:47.100296+0000","last_became_active":"2024-09-16T02:03:34.741838+0000","last_became_peered":"2024-09-16T02:03:34.741838+0000","last_unstale":"2024-09-16T02:03:47.100296+0000","last_undegraded":"2024-09-16T02:03:47.100296+0000","last_fullsized":"2024-09-16T02:03:47.100296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:57.338380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54508,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":120,"num_read_kb":119,"num_write":161,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":163524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678035,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":405846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1047832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037788,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":406082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266723,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":406002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495655,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1445888,"data_stored":998327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659180,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":405471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888112,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1458176,"data_stored":997618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051638,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":406301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280572,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":405782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476801,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":457119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607623,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":455376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":406742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":50112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:04:07.645 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T02:04:07.646 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T02:04:07.646 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T02:04:07.946 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:07 smithi172 ceph-mon[35340]: pgmap v296: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-16T02:04:08.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:07 smithi103 ceph-mon[30398]: pgmap v296: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-16T02:04:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:08.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:07 smithi136 ceph-mon[35323]: pgmap v296: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-16T02:04:08.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.542+0000 7fd741600640 1 -- 172.21.15.103:0/1042416999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73c104e50 msgr2=0x7fd73c1052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:08.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.542+0000 7fd741600640 1 --2- 172.21.15.103:0/1042416999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73c104e50 0x7fd73c1052d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd728009900 tx=0x7fd72802a3f0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.542+0000 7fd741600640 1 -- 172.21.15.103:0/1042416999 shutdown_connections 2024-09-16T02:04:08.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.542+0000 7fd741600640 1 --2- 172.21.15.103:0/1042416999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c105810 0x7fd73c10c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.542+0000 7fd741600640 1 --2- 172.21.15.103:0/1042416999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73c104e50 0x7fd73c1052d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.542+0000 7fd741600640 1 --2- 172.21.15.103:0/1042416999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd73c103c50 0x7fd73c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.542+0000 7fd741600640 1 -- 172.21.15.103:0/1042416999 >> 172.21.15.103:0/1042416999 conn(0x7fd73c0ff3e0 msgr2=0x7fd73c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:08.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.542+0000 7fd741600640 1 -- 172.21.15.103:0/1042416999 shutdown_connections 2024-09-16T02:04:08.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.542+0000 7fd741600640 1 -- 172.21.15.103:0/1042416999 wait complete. 2024-09-16T02:04:08.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.543+0000 7fd741600640 1 Processor -- start 2024-09-16T02:04:08.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.543+0000 7fd741600640 1 -- start start 2024-09-16T02:04:08.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.543+0000 7fd741600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c103c50 0x7fd73c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:08.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd741600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd73c104e50 0x7fd73c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:08.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd741600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73c105810 0x7fd73c1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd741600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c10f900 con 0x7fd73c105810 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd741600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c10f780 con 0x7fd73c103c50 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd741600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c10fa80 con 0x7fd73c104e50 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd740c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73c105810 0x7fd73c1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd73be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c103c50 0x7fd73c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd73be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c103c50 0x7fd73c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:59836/0 (socket says 172.21.15.103:59836) 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd73be00640 1 -- 172.21.15.103:0/122180650 learned_addr learned my addr 172.21.15.103:0/122180650 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd73b400640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd73c104e50 0x7fd73c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd740c00640 1 -- 172.21.15.103:0/122180650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd73c104e50 msgr2=0x7fd73c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd740c00640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd73c104e50 0x7fd73c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd740c00640 1 -- 172.21.15.103:0/122180650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c103c50 msgr2=0x7fd73c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd740c00640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c103c50 0x7fd73c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd740c00640 1 -- 172.21.15.103:0/122180650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c1a1b80 con 0x7fd73c105810 2024-09-16T02:04:08.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.544+0000 7fd73be00640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c103c50 0x7fd73c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:08.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.545+0000 7fd740c00640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73c105810 0x7fd73c1a1510 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd73400da10 tx=0x7fd73400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:08.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.545+0000 7fd738c00640 1 -- 172.21.15.103:0/122180650 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73400bd70 con 0x7fd73c105810 2024-09-16T02:04:08.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.545+0000 7fd738c00640 1 -- 172.21.15.103:0/122180650 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd734003e50 con 0x7fd73c105810 2024-09-16T02:04:08.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.545+0000 7fd741600640 1 -- 172.21.15.103:0/122180650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c1a1e70 con 0x7fd73c105810 2024-09-16T02:04:08.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.545+0000 7fd738c00640 1 -- 172.21.15.103:0/122180650 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7340106a0 con 0x7fd73c105810 2024-09-16T02:04:08.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.545+0000 7fd741600640 1 -- 172.21.15.103:0/122180650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c1a2320 con 0x7fd73c105810 2024-09-16T02:04:08.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.546+0000 7fd741600640 1 -- 172.21.15.103:0/122180650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c10b070 con 0x7fd73c105810 2024-09-16T02:04:08.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.548+0000 7fd738c00640 1 -- 172.21.15.103:0/122180650 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd734003970 con 0x7fd73c105810 2024-09-16T02:04:08.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.549+0000 7fd738c00640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd7200a3380 0x7fd7200a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:08.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.549+0000 7fd738c00640 1 -- 172.21.15.103:0/122180650 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fd7340cfce0 con 0x7fd73c105810 2024-09-16T02:04:08.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.549+0000 7fd73be00640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd7200a3380 0x7fd7200a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:08.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.550+0000 7fd73be00640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd7200a3380 0x7fd7200a5840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0104e0 tx=0x7fd72c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:08.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.553+0000 7fd738c00640 1 -- 172.21.15.103:0/122180650 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd73409c950 con 0x7fd73c105810 2024-09-16T02:04:08.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:08 smithi103 ceph-mon[30398]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:08.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.797+0000 7fd741600640 1 -- 172.21.15.103:0/122180650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd73c02d9c0 con 0x7fd73c105810 2024-09-16T02:04:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.800+0000 7fd738c00640 1 -- 172.21.15.103:0/122180650 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd73409c250 con 0x7fd73c105810 2024-09-16T02:04:08.801 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2024-09-16T02:04:08.801 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAYkudmO9KaLxAAggMT6uFuFOKo7ocXIY03MA== 2024-09-16T02:04:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 -- 172.21.15.103:0/122180650 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd7200a3380 msgr2=0x7fd7200a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd7200a3380 0x7fd7200a5840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0104e0 tx=0x7fd72c00b410 comp rx=0 tx=0).stop 2024-09-16T02:04:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 -- 172.21.15.103:0/122180650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73c105810 msgr2=0x7fd73c1a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73c105810 0x7fd73c1a1510 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd73400da10 tx=0x7fd73400ded0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 -- 172.21.15.103:0/122180650 shutdown_connections 2024-09-16T02:04:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd7200a3380 0x7fd7200a5840 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73c105810 0x7fd73c1a1510 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd73c104e50 0x7fd73c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 --2- 172.21.15.103:0/122180650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd73c103c50 0x7fd73c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 -- 172.21.15.103:0/122180650 >> 172.21.15.103:0/122180650 conn(0x7fd73c0ff3e0 msgr2=0x7fd73c108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 -- 172.21.15.103:0/122180650 shutdown_connections 2024-09-16T02:04:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:08.803+0000 7fd741600640 1 -- 172.21.15.103:0/122180650 wait complete. 2024-09-16T02:04:09.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:08 smithi136 ceph-mon[35323]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:09.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:08 smithi172 ceph-mon[35340]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:09.633 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T02:04:09.633 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T02:04:09.633 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T02:04:09.713 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T02:04:09.713 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T02:04:09.713 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T02:04:09.925 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:09 smithi136 ceph-mon[35323]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-09-16T02:04:09.925 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:09 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/122180650' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T02:04:09.925 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:09 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/122180650' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T02:04:10.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:09 smithi103 ceph-mon[30398]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-09-16T02:04:10.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:09 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/122180650' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T02:04:10.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:09 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/122180650' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T02:04:10.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:09 smithi172 ceph-mon[35340]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-09-16T02:04:10.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:09 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/122180650' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T02:04:10.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:09 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/122180650' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T02:04:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:10.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.643+0000 7ffb3d200640 1 -- 172.21.15.103:0/1834797099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38108f60 msgr2=0x7ffb38110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.643+0000 7ffb3d200640 1 --2- 172.21.15.103:0/1834797099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38108f60 0x7ffb38110ae0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009900 tx=0x7ffb3002a410 comp rx=0 tx=0).stop 2024-09-16T02:04:10.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.643+0000 7ffb3d200640 1 -- 172.21.15.103:0/1834797099 shutdown_connections 2024-09-16T02:04:10.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.643+0000 7ffb3d200640 1 --2- 172.21.15.103:0/1834797099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38108f60 0x7ffb38110ae0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.643+0000 7ffb3d200640 1 --2- 172.21.15.103:0/1834797099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb381047e0 0x7ffb38108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.643+0000 7ffb3d200640 1 --2- 172.21.15.103:0/1834797099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb38103e30 0x7ffb38104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.643+0000 7ffb3d200640 1 -- 172.21.15.103:0/1834797099 >> 172.21.15.103:0/1834797099 conn(0x7ffb380ff910 msgr2=0x7ffb38101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.643+0000 7ffb3d200640 1 -- 172.21.15.103:0/1834797099 shutdown_connections 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.643+0000 7ffb3d200640 1 -- 172.21.15.103:0/1834797099 wait complete. 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.644+0000 7ffb3d200640 1 Processor -- start 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.644+0000 7ffb3d200640 1 -- start start 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb3d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb38103e30 0x7ffb3819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb3d200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb381047e0 0x7ffb3819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb3d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38108f60 0x7ffb3819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb3d200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb38113fc0 con 0x7ffb38108f60 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb3d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffb38113e40 con 0x7ffb38103e30 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb3d200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffb38114140 con 0x7ffb381047e0 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb37e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38108f60 0x7ffb3819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb37e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38108f60 0x7ffb3819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51586/0 (socket says 172.21.15.103:51586) 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb2ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb381047e0 0x7ffb3819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb37e00640 1 -- 172.21.15.103:0/3524442945 learned_addr learned my addr 172.21.15.103:0/3524442945 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb37400640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb38103e30 0x7ffb3819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb37e00640 1 -- 172.21.15.103:0/3524442945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb381047e0 msgr2=0x7ffb3819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb37e00640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb381047e0 0x7ffb3819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb37e00640 1 -- 172.21.15.103:0/3524442945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb38103e30 msgr2=0x7ffb3819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb37e00640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb38103e30 0x7ffb3819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb37e00640 1 -- 172.21.15.103:0/3524442945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3819f3b0 con 0x7ffb38108f60 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.645+0000 7ffb37400640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb38103e30 0x7ffb3819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.646+0000 7ffb37e00640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38108f60 0x7ffb3819ecd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb30005ab0 tx=0x7ffb30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.646+0000 7ffb34c00640 1 -- 172.21.15.103:0/3524442945 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3003f030 con 0x7ffb38108f60 2024-09-16T02:04:10.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.646+0000 7ffb34c00640 1 -- 172.21.15.103:0/3524442945 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb30003ad0 con 0x7ffb38108f60 2024-09-16T02:04:10.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.646+0000 7ffb3d200640 1 -- 172.21.15.103:0/3524442945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3819f640 con 0x7ffb38108f60 2024-09-16T02:04:10.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.646+0000 7ffb34c00640 1 -- 172.21.15.103:0/3524442945 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30035070 con 0x7ffb38108f60 2024-09-16T02:04:10.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.646+0000 7ffb3d200640 1 -- 172.21.15.103:0/3524442945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb381a6f80 con 0x7ffb38108f60 2024-09-16T02:04:10.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.647+0000 7ffb3d200640 1 -- 172.21.15.103:0/3524442945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb38105310 con 0x7ffb38108f60 2024-09-16T02:04:10.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.649+0000 7ffb34c00640 1 -- 172.21.15.103:0/3524442945 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ffb30042030 con 0x7ffb38108f60 2024-09-16T02:04:10.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.650+0000 7ffb34c00640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb1c0a32d0 0x7ffb1c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:10.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.650+0000 7ffb34c00640 1 -- 172.21.15.103:0/3524442945 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7ffb300f0290 con 0x7ffb38108f60 2024-09-16T02:04:10.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.650+0000 7ffb37400640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb1c0a32d0 0x7ffb1c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:10.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.651+0000 7ffb37400640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb1c0a32d0 0x7ffb1c0a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffb28005fd0 tx=0x7ffb28009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:10.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.654+0000 7ffb34c00640 1 -- 172.21.15.103:0/3524442945 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb300bc8b0 con 0x7ffb38108f60 2024-09-16T02:04:10.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.874+0000 7ffb3d200640 1 -- 172.21.15.103:0/3524442945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ffb38104210 con 0x7ffb38108f60 2024-09-16T02:04:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.879+0000 7ffb34c00640 1 -- 172.21.15.103:0/3524442945 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264056 (secure 0 0 0) 0x7ffb38104210 con 0x7ffb38108f60 2024-09-16T02:04:10.884 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:10.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 -- 172.21.15.103:0/3524442945 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb1c0a32d0 msgr2=0x7ffb1c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb1c0a32d0 0x7ffb1c0a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffb28005fd0 tx=0x7ffb28009500 comp rx=0 tx=0).stop 2024-09-16T02:04:10.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 -- 172.21.15.103:0/3524442945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38108f60 msgr2=0x7ffb3819ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:10.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38108f60 0x7ffb3819ecd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb30005ab0 tx=0x7ffb30004270 comp rx=0 tx=0).stop 2024-09-16T02:04:10.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 -- 172.21.15.103:0/3524442945 shutdown_connections 2024-09-16T02:04:10.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ffb1c0a32d0 0x7ffb1c0a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb38108f60 0x7ffb3819ecd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb381047e0 0x7ffb3819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 --2- 172.21.15.103:0/3524442945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb38103e30 0x7ffb3819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 -- 172.21.15.103:0/3524442945 >> 172.21.15.103:0/3524442945 conn(0x7ffb380ff910 msgr2=0x7ffb38100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 -- 172.21.15.103:0/3524442945 shutdown_connections 2024-09-16T02:04:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:10.883+0000 7ffb3d200640 1 -- 172.21.15.103:0/3524442945 wait complete. 2024-09-16T02:04:11.601 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3075264030},{"type":"v1","addr":"172.21.15.103:6801","nonce":3075264030}]},"active_addr":"172.21.15.103:6801/3075264030","active_change":"2024-09-16T01:56:28.360056+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.103:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":333054008}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":945987873}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2356855531}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":846360008}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":1043488244}]}]} 2024-09-16T02:04:11.606 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T02:04:11.606 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T02:04:11.606 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:04:11.976 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:12.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:11 smithi103 ceph-mon[30398]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-16T02:04:12.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:11 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3524442945' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T02:04:12.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:11 smithi136 ceph-mon[35323]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-16T02:04:12.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:11 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3524442945' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T02:04:12.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:11 smithi172 ceph-mon[35340]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-16T02:04:12.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:11 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3524442945' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T02:04:12.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.504+0000 7f7017200640 1 -- 172.21.15.103:0/262069987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010106f10 msgr2=0x7f701010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:12.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.504+0000 7f7017200640 1 --2- 172.21.15.103:0/262069987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010106f10 0x7f701010ea90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7008009ca0 tx=0x7f700802a8c0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.505+0000 7f7017200640 1 -- 172.21.15.103:0/262069987 shutdown_connections 2024-09-16T02:04:12.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.505+0000 7f7017200640 1 --2- 172.21.15.103:0/262069987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010106f10 0x7f701010ea90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.505+0000 7f7017200640 1 --2- 172.21.15.103:0/262069987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010102790 0x7f70101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.505+0000 7f7017200640 1 --2- 172.21.15.103:0/262069987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7010101de0 0x7f70101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.505+0000 7f7017200640 1 -- 172.21.15.103:0/262069987 >> 172.21.15.103:0/262069987 conn(0x7f70100fd8c0 msgr2=0x7f70100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:12.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.505+0000 7f7017200640 1 -- 172.21.15.103:0/262069987 shutdown_connections 2024-09-16T02:04:12.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.505+0000 7f7017200640 1 -- 172.21.15.103:0/262069987 wait complete. 2024-09-16T02:04:12.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f7017200640 1 Processor -- start 2024-09-16T02:04:12.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f7017200640 1 -- start start 2024-09-16T02:04:12.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f7017200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010101de0 0x7f701019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:12.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f7017200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010102790 0x7f701019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:12.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f7017200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7010106f10 0x7f701019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:12.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f7017200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7010111fc0 con 0x7f7010101de0 2024-09-16T02:04:12.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f7017200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7010111e40 con 0x7f7010102790 2024-09-16T02:04:12.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f7017200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7010112140 con 0x7f7010106f10 2024-09-16T02:04:12.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f700fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010101de0 0x7f701019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:12.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f700fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010101de0 0x7f701019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51616/0 (socket says 172.21.15.103:51616) 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.506+0000 7f700fe00640 1 -- 172.21.15.103:0/1450723806 learned_addr learned my addr 172.21.15.103:0/1450723806 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.507+0000 7f7015200640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7010106f10 0x7f701019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.507+0000 7f7007e00640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010102790 0x7f701019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.507+0000 7f700fe00640 1 -- 172.21.15.103:0/1450723806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7010106f10 msgr2=0x7f701019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.507+0000 7f700fe00640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7010106f10 0x7f701019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.507+0000 7f700fe00640 1 -- 172.21.15.103:0/1450723806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010102790 msgr2=0x7f701019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.507+0000 7f700fe00640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010102790 0x7f701019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.507+0000 7f700fe00640 1 -- 172.21.15.103:0/1450723806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701019f540 con 0x7f7010101de0 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.507+0000 7f7015200640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7010106f10 0x7f701019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.507+0000 7f700fe00640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010101de0 0x7f701019a590 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f700000e9f0 tx=0x7f700000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.508+0000 7f700d600640 1 -- 172.21.15.103:0/1450723806 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700000cd80 con 0x7f7010101de0 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.508+0000 7f700d600640 1 -- 172.21.15.103:0/1450723806 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7000005500 con 0x7f7010101de0 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.508+0000 7f700d600640 1 -- 172.21.15.103:0/1450723806 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000010690 con 0x7f7010101de0 2024-09-16T02:04:12.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.508+0000 7f7017200640 1 -- 172.21.15.103:0/1450723806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701019f830 con 0x7f7010101de0 2024-09-16T02:04:12.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.508+0000 7f7017200640 1 -- 172.21.15.103:0/1450723806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70101a7110 con 0x7f7010101de0 2024-09-16T02:04:12.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.509+0000 7f7017200640 1 -- 172.21.15.103:0/1450723806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70101021c0 con 0x7f7010101de0 2024-09-16T02:04:12.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.511+0000 7f700d600640 1 -- 172.21.15.103:0/1450723806 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f70000046c0 con 0x7f7010101de0 2024-09-16T02:04:12.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.513+0000 7f700d600640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6ff40a32d0 0x7f6ff40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:12.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.513+0000 7f700d600640 1 -- 172.21.15.103:0/1450723806 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f70000cff50 con 0x7f7010101de0 2024-09-16T02:04:12.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.513+0000 7f7007e00640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6ff40a32d0 0x7f6ff40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:12.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.514+0000 7f7007e00640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6ff40a32d0 0x7f6ff40a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7010076480 tx=0x7f6ff800c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:12.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.516+0000 7f700d600640 1 -- 172.21.15.103:0/1450723806 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f700009cbc0 con 0x7f7010101de0 2024-09-16T02:04:12.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.701+0000 7f7017200640 1 -- 172.21.15.103:0/1450723806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f701019b8f0 con 0x7f7010101de0 2024-09-16T02:04:12.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.703+0000 7f700d600640 1 -- 172.21.15.103:0/1450723806 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22361 (secure 0 0 0) 0x7f700009c4c0 con 0x7f7010101de0 2024-09-16T02:04:12.706 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:12.706 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":77,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:03:37.830327+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.706+0000 7f7017200640 1 -- 172.21.15.103:0/1450723806 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6ff40a32d0 msgr2=0x7f6ff40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.706+0000 7f7017200640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6ff40a32d0 0x7f6ff40a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7010076480 tx=0x7f6ff800c4a0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.706+0000 7f7017200640 1 -- 172.21.15.103:0/1450723806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010101de0 msgr2=0x7f701019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.706+0000 7f7017200640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010101de0 0x7f701019a590 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f700000e9f0 tx=0x7f700000edf0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.707+0000 7f7017200640 1 -- 172.21.15.103:0/1450723806 shutdown_connections 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.707+0000 7f7017200640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6ff40a32d0 0x7f6ff40a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.707+0000 7f7017200640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7010106f10 0x7f701019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.707+0000 7f7017200640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010102790 0x7f701019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.707+0000 7f7017200640 1 --2- 172.21.15.103:0/1450723806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010101de0 0x7f701019a590 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.707+0000 7f7017200640 1 -- 172.21.15.103:0/1450723806 >> 172.21.15.103:0/1450723806 conn(0x7f70100fd8c0 msgr2=0x7f70100fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.707+0000 7f7017200640 1 -- 172.21.15.103:0/1450723806 shutdown_connections 2024-09-16T02:04:12.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:12.707+0000 7f7017200640 1 -- 172.21.15.103:0/1450723806 wait complete. 2024-09-16T02:04:13.468 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T02:04:13.468 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:04:13.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:13 smithi103 ceph-mon[30398]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T02:04:13.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:13 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1450723806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:13.854 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:14.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:13 smithi136 ceph-mon[35323]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T02:04:14.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:13 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1450723806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:14.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:13 smithi172 ceph-mon[35340]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T02:04:14.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:13 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1450723806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:14.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.434+0000 7f16af200640 1 -- 172.21.15.103:0/3579465312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a81079c0 msgr2=0x7f16a8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:14.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.434+0000 7f16af200640 1 --2- 172.21.15.103:0/3579465312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a81079c0 0x7f16a8107da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f169c009930 tx=0x7f169c02a420 comp rx=0 tx=0).stop 2024-09-16T02:04:14.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.434+0000 7f16af200640 1 -- 172.21.15.103:0/3579465312 shutdown_connections 2024-09-16T02:04:14.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.434+0000 7f16af200640 1 --2- 172.21.15.103:0/3579465312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8102360 0x7f16a810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.434+0000 7f16af200640 1 --2- 172.21.15.103:0/3579465312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a81019c0 0x7f16a8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.434+0000 7f16af200640 1 --2- 172.21.15.103:0/3579465312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a81079c0 0x7f16a8107da0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.434+0000 7f16af200640 1 -- 172.21.15.103:0/3579465312 >> 172.21.15.103:0/3579465312 conn(0x7f16a80fd740 msgr2=0x7f16a80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:14.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.434+0000 7f16af200640 1 -- 172.21.15.103:0/3579465312 shutdown_connections 2024-09-16T02:04:14.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.434+0000 7f16af200640 1 -- 172.21.15.103:0/3579465312 wait complete. 2024-09-16T02:04:14.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.436+0000 7f16af200640 1 Processor -- start 2024-09-16T02:04:14.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.436+0000 7f16af200640 1 -- start start 2024-09-16T02:04:14.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.436+0000 7f16af200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a81019c0 0x7f16a819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:14.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.436+0000 7f16af200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8102360 0x7f16a819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:14.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.436+0000 7f16af200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a81079c0 0x7f16a819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:14.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.436+0000 7f16af200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8111dc0 con 0x7f16a81079c0 2024-09-16T02:04:14.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.436+0000 7f16af200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8111c40 con 0x7f16a81019c0 2024-09-16T02:04:14.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.436+0000 7f16af200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8111f40 con 0x7f16a8102360 2024-09-16T02:04:14.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.436+0000 7f16ae800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a81079c0 0x7f16a819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:14.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.437+0000 7f16ae800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a81079c0 0x7f16a819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51634/0 (socket says 172.21.15.103:51634) 2024-09-16T02:04:14.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.437+0000 7f16ae800640 1 -- 172.21.15.103:0/2337807964 learned_addr learned my addr 172.21.15.103:0/2337807964 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.437+0000 7f16ade00640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a81019c0 0x7f16a819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.437+0000 7f16ad400640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8102360 0x7f16a819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.437+0000 7f16ae800640 1 -- 172.21.15.103:0/2337807964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8102360 msgr2=0x7f16a819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.437+0000 7f16ae800640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8102360 0x7f16a819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.437+0000 7f16ae800640 1 -- 172.21.15.103:0/2337807964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a81019c0 msgr2=0x7f16a819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.437+0000 7f16ae800640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a81019c0 0x7f16a819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.437+0000 7f16ae800640 1 -- 172.21.15.103:0/2337807964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a819f450 con 0x7f16a81079c0 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.437+0000 7f16ae800640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a81079c0 0x7f16a819ed70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16a400eaa0 tx=0x7f16a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.438+0000 7f16a2a00640 1 -- 172.21.15.103:0/2337807964 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a400cdb0 con 0x7f16a81079c0 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.438+0000 7f16a2a00640 1 -- 172.21.15.103:0/2337807964 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4005020 con 0x7f16a81079c0 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.438+0000 7f16af200640 1 -- 172.21.15.103:0/2337807964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a819f740 con 0x7f16a81079c0 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.438+0000 7f16a2a00640 1 -- 172.21.15.103:0/2337807964 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a40116b0 con 0x7f16a81079c0 2024-09-16T02:04:14.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.438+0000 7f16af200640 1 -- 172.21.15.103:0/2337807964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a81a7020 con 0x7f16a81079c0 2024-09-16T02:04:14.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.439+0000 7f16af200640 1 -- 172.21.15.103:0/2337807964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a8101e20 con 0x7f16a81079c0 2024-09-16T02:04:14.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.441+0000 7f16a2a00640 1 -- 172.21.15.103:0/2337807964 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f16a4005560 con 0x7f16a81079c0 2024-09-16T02:04:14.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.442+0000 7f16a2a00640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f16900a3350 0x7f16900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:14.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.442+0000 7f16a2a00640 1 -- 172.21.15.103:0/2337807964 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f16a40cfac0 con 0x7f16a81079c0 2024-09-16T02:04:14.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.442+0000 7f16ade00640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f16900a3350 0x7f16900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:14.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.443+0000 7f16ade00640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f16900a3350 0x7f16900a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f169c009900 tx=0x7f169c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:14.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.446+0000 7f16a2a00640 1 -- 172.21.15.103:0/2337807964 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16a4016030 con 0x7f16a81079c0 2024-09-16T02:04:14.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.622+0000 7f16af200640 1 -- 172.21.15.103:0/2337807964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16a81a7350 con 0x7f16a81079c0 2024-09-16T02:04:14.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.624+0000 7f16a2a00640 1 -- 172.21.15.103:0/2337807964 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22361 (secure 0 0 0) 0x7f16a409c0e0 con 0x7f16a81079c0 2024-09-16T02:04:14.626 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:14.627 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":77,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:03:37.830327+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:04:14.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.626+0000 7f16af200640 1 -- 172.21.15.103:0/2337807964 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f16900a3350 msgr2=0x7f16900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:14.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.626+0000 7f16af200640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f16900a3350 0x7f16900a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f169c009900 tx=0x7f169c0058d0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.626+0000 7f16af200640 1 -- 172.21.15.103:0/2337807964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a81079c0 msgr2=0x7f16a819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:14.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.626+0000 7f16af200640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a81079c0 0x7f16a819ed70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16a400eaa0 tx=0x7f16a400ef60 comp rx=0 tx=0).stop 2024-09-16T02:04:14.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.626+0000 7f16af200640 1 -- 172.21.15.103:0/2337807964 shutdown_connections 2024-09-16T02:04:14.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.626+0000 7f16af200640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f16900a3350 0x7f16900a5810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.626+0000 7f16af200640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a81079c0 0x7f16a819ed70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.626+0000 7f16af200640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8102360 0x7f16a819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.626+0000 7f16af200640 1 --2- 172.21.15.103:0/2337807964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a81019c0 0x7f16a819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:14.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.626+0000 7f16af200640 1 -- 172.21.15.103:0/2337807964 >> 172.21.15.103:0/2337807964 conn(0x7f16a80fd740 msgr2=0x7f16a810d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:14.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.627+0000 7f16af200640 1 -- 172.21.15.103:0/2337807964 shutdown_connections 2024-09-16T02:04:14.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:14.627+0000 7f16af200640 1 -- 172.21.15.103:0/2337807964 wait complete. 2024-09-16T02:04:15.499 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T02:04:15.500 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T02:04:15.500 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T02:04:15.500 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T02:04:15.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T02:04:15.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T02:04:15.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T02:04:15.503 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T02:04:15.503 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-16T02:04:15.504 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-16T02:04:15.504 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-16T02:04:15.505 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-16T02:04:15.725 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:15 smithi103 ceph-mon[30398]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-16T02:04:15.726 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:15 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2337807964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:16.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:15 smithi136 ceph-mon[35323]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-16T02:04:16.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:15 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2337807964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:15 smithi172 ceph-mon[35340]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-16T02:04:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:15 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2337807964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:16.427 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:16.529 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:16.550 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.440+0000 7f1c67e00640 1 -- 172.21.15.103:0/1388102887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c680745c0 msgr2=0x7f1c680749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.440+0000 7f1c67e00640 1 --2- 172.21.15.103:0/1388102887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c680745c0 0x7f1c680749a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009900 tx=0x7f1c5802a480 comp rx=0 tx=0).stop 2024-09-16T02:04:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.440+0000 7f1c67e00640 1 -- 172.21.15.103:0/1388102887 shutdown_connections 2024-09-16T02:04:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.440+0000 7f1c67e00640 1 --2- 172.21.15.103:0/1388102887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c6810e260 0x7f1c68110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.440+0000 7f1c67e00640 1 --2- 172.21.15.103:0/1388102887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c68074ee0 0x7f1c6810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.440+0000 7f1c67e00640 1 --2- 172.21.15.103:0/1388102887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c680745c0 0x7f1c680749a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.440+0000 7f1c67e00640 1 -- 172.21.15.103:0/1388102887 >> 172.21.15.103:0/1388102887 conn(0x7f1c680fd6e0 msgr2=0x7f1c680ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.440+0000 7f1c67e00640 1 -- 172.21.15.103:0/1388102887 shutdown_connections 2024-09-16T02:04:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.440+0000 7f1c67e00640 1 -- 172.21.15.103:0/1388102887 wait complete. 2024-09-16T02:04:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.441+0000 7f1c67e00640 1 Processor -- start 2024-09-16T02:04:17.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.441+0000 7f1c67e00640 1 -- start start 2024-09-16T02:04:17.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.441+0000 7f1c67e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c680745c0 0x7f1c6819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.441+0000 7f1c67e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c68074ee0 0x7f1c6819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.441+0000 7f1c67e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c6810e260 0x7f1c6819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.441+0000 7f1c67e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c68114060 con 0x7f1c68074ee0 2024-09-16T02:04:17.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c67e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1c68113ee0 con 0x7f1c6810e260 2024-09-16T02:04:17.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c67e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c681141e0 con 0x7f1c680745c0 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c5e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c68074ee0 0x7f1c6819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c5e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c68074ee0 0x7f1c6819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51648/0 (socket says 172.21.15.103:51648) 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c5e000640 1 -- 172.21.15.103:0/3420976444 learned_addr learned my addr 172.21.15.103:0/3420976444 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c66a00640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c680745c0 0x7f1c6819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c67400640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c6810e260 0x7f1c6819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c5e000640 1 -- 172.21.15.103:0/3420976444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c680745c0 msgr2=0x7f1c6819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c5e000640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c680745c0 0x7f1c6819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c5e000640 1 -- 172.21.15.103:0/3420976444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c6810e260 msgr2=0x7f1c6819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c5e000640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c6810e260 0x7f1c6819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.442+0000 7f1c5e000640 1 -- 172.21.15.103:0/3420976444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6819f4e0 con 0x7f1c68074ee0 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.443+0000 7f1c5e000640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c68074ee0 0x7f1c6819aa70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000b4a0 tx=0x7f1c5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.443+0000 7f1c5fe00640 1 -- 172.21.15.103:0/3420976444 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50014070 con 0x7f1c68074ee0 2024-09-16T02:04:17.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.443+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6819f7d0 con 0x7f1c68074ee0 2024-09-16T02:04:17.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.443+0000 7f1c5fe00640 1 -- 172.21.15.103:0/3420976444 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c50010af0 con 0x7f1c68074ee0 2024-09-16T02:04:17.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.443+0000 7f1c5fe00640 1 -- 172.21.15.103:0/3420976444 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50017d10 con 0x7f1c68074ee0 2024-09-16T02:04:17.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.443+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c681a70b0 con 0x7f1c68074ee0 2024-09-16T02:04:17.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.444+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c68062980 con 0x7f1c68074ee0 2024-09-16T02:04:17.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.446+0000 7f1c5fe00640 1 -- 172.21.15.103:0/3420976444 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1c50017070 con 0x7f1c68074ee0 2024-09-16T02:04:17.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.447+0000 7f1c5fe00640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1c4c0a3350 0x7f1c4c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.447+0000 7f1c5fe00640 1 -- 172.21.15.103:0/3420976444 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f1c500ceb20 con 0x7f1c68074ee0 2024-09-16T02:04:17.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.448+0000 7f1c66a00640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1c4c0a3350 0x7f1c4c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.448+0000 7f1c5fe00640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] conn(0x7f1c4c0acba0 0x7f1c4c0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.448+0000 7f1c5fe00640 1 -- 172.21.15.103:0/3420976444 --> [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c4c0af670 con 0x7f1c4c0acba0 2024-09-16T02:04:17.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.448+0000 7f1c5fe00640 1 -- 172.21.15.103:0/3420976444 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f1c500d32a0 con 0x7f1c68074ee0 2024-09-16T02:04:17.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.448+0000 7f1c66a00640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1c4c0a3350 0x7f1c4c0a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c58000c00 tx=0x7f1c58005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.450+0000 7f1c67400640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] conn(0x7f1c4c0acba0 0x7f1c4c0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.452+0000 7f1c67400640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] conn(0x7f1c4c0acba0 0x7f1c4c0aefc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.453+0000 7f1c5fe00640 1 -- 172.21.15.103:0/3420976444 <== osd.10 v2:172.21.15.136:6824/3121812139 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c4c0af670 con 0x7f1c4c0acba0 2024-09-16T02:04:17.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.469+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 --> [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1c68062db0 con 0x7f1c4c0acba0 2024-09-16T02:04:17.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.470+0000 7f1c5fe00640 1 -- 172.21.15.103:0/3420976444 <== osd.10 v2:172.21.15.136:6824/3121812139 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1c68062db0 con 0x7f1c4c0acba0 2024-09-16T02:04:17.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.470+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 >> [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] conn(0x7f1c4c0acba0 msgr2=0x7f1c4c0aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.470+0000 7f1c67e00640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] conn(0x7f1c4c0acba0 0x7f1c4c0aefc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.470+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1c4c0a3350 msgr2=0x7f1c4c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.470+0000 7f1c67e00640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1c4c0a3350 0x7f1c4c0a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c58000c00 tx=0x7f1c58005b50 comp rx=0 tx=0).stop 2024-09-16T02:04:17.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.470+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c68074ee0 msgr2=0x7f1c6819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.470+0000 7f1c67e00640 1 --2- 172.21.15.103:0/3420976444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c68074ee0 0x7f1c6819aa70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000b4a0 tx=0x7f1c5000b960 comp rx=0 tx=0).stop 2024-09-16T02:04:17.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.470+0000 7f1c67400640 1 -- 172.21.15.103:0/3420976444 reap_dead start 2024-09-16T02:04:17.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.470+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 shutdown_connections 2024-09-16T02:04:17.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.470+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 >> 172.21.15.103:0/3420976444 conn(0x7f1c680fd6e0 msgr2=0x7f1c680fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:17.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.471+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 shutdown_connections 2024-09-16T02:04:17.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.471+0000 7f1c67e00640 1 -- 172.21.15.103:0/3420976444 wait complete. 2024-09-16T02:04:17.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.658+0000 7f457ce00640 1 -- 172.21.15.103:0/554852285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4578105d90 msgr2=0x7f4578106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.658+0000 7f457ce00640 1 --2- 172.21.15.103:0/554852285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4578105d90 0x7f4578106220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4570009ca0 tx=0x7f457002a940 comp rx=0 tx=0).stop 2024-09-16T02:04:17.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.658+0000 7f457ce00640 1 -- 172.21.15.103:0/554852285 shutdown_connections 2024-09-16T02:04:17.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.658+0000 7f457ce00640 1 --2- 172.21.15.103:0/554852285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4578105d90 0x7f4578106220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.658+0000 7f457ce00640 1 --2- 172.21.15.103:0/554852285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45781012a0 0x7f4578105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.658+0000 7f457ce00640 1 --2- 172.21.15.103:0/554852285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4578100980 0x7f4578100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.658+0000 7f457ce00640 1 -- 172.21.15.103:0/554852285 >> 172.21.15.103:0/554852285 conn(0x7f45780fc610 msgr2=0x7f45780fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:17.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.658+0000 7f457ce00640 1 -- 172.21.15.103:0/554852285 shutdown_connections 2024-09-16T02:04:17.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.658+0000 7f457ce00640 1 -- 172.21.15.103:0/554852285 wait complete. 2024-09-16T02:04:17.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.659+0000 7f457ce00640 1 Processor -- start 2024-09-16T02:04:17.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.659+0000 7f457ce00640 1 -- start start 2024-09-16T02:04:17.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f457ce00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4578100980 0x7f45781a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f457ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45781012a0 0x7f45781a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f457ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4578105d90 0x7f457819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f457ce00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4578118420 con 0x7f45781012a0 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f457ce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f45781182a0 con 0x7f4578105d90 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f457ce00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f45781185a0 con 0x7f4578100980 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f4576a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45781012a0 0x7f45781a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f4576a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45781012a0 0x7f45781a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51670/0 (socket says 172.21.15.103:51670) 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f4576a00640 1 -- 172.21.15.103:0/412264614 learned_addr learned my addr 172.21.15.103:0/412264614 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f4577400640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4578100980 0x7f45781a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f4577e00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4578105d90 0x7f457819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f4576a00640 1 -- 172.21.15.103:0/412264614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4578100980 msgr2=0x7f45781a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f4576a00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4578100980 0x7f45781a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f4576a00640 1 -- 172.21.15.103:0/412264614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4578105d90 msgr2=0x7f457819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f4576a00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4578105d90 0x7f457819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.660+0000 7f4576a00640 1 -- 172.21.15.103:0/412264614 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457819ef30 con 0x7f45781012a0 2024-09-16T02:04:17.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.661+0000 7f4576a00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45781012a0 0x7f45781a4b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f456400e9f0 tx=0x7f456400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.661+0000 7f456fe00640 1 -- 172.21.15.103:0/412264614 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456400ccc0 con 0x7f45781012a0 2024-09-16T02:04:17.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.661+0000 7f456fe00640 1 -- 172.21.15.103:0/412264614 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456400ce60 con 0x7f45781012a0 2024-09-16T02:04:17.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.661+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457819f220 con 0x7f45781012a0 2024-09-16T02:04:17.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.661+0000 7f456fe00640 1 -- 172.21.15.103:0/412264614 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564010690 con 0x7f45781012a0 2024-09-16T02:04:17.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.661+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45781ab570 con 0x7f45781012a0 2024-09-16T02:04:17.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.662+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4578105140 con 0x7f45781012a0 2024-09-16T02:04:17.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.665+0000 7f456fe00640 1 -- 172.21.15.103:0/412264614 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f45640046c0 con 0x7f45781012a0 2024-09-16T02:04:17.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.666+0000 7f456fe00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f455c0a3350 0x7f455c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.666+0000 7f456fe00640 1 -- 172.21.15.103:0/412264614 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f45640d0120 con 0x7f45781012a0 2024-09-16T02:04:17.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.666+0000 7f4577400640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f455c0a3350 0x7f455c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.666+0000 7f456fe00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] conn(0x7f455c0acba0 0x7f455c0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:17.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.666+0000 7f456fe00640 1 -- 172.21.15.103:0/412264614 --> [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f455c0af670 con 0x7f455c0acba0 2024-09-16T02:04:17.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.666+0000 7f456fe00640 1 -- 172.21.15.103:0/412264614 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f45640d04d0 con 0x7f45781012a0 2024-09-16T02:04:17.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.667+0000 7f4577e00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] conn(0x7f455c0acba0 0x7f455c0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:17.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.667+0000 7f4577400640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f455c0a3350 0x7f455c0a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4568009bd0 tx=0x7f4568005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.667+0000 7f4577e00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] conn(0x7f455c0acba0 0x7f455c0aefc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:17.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.668+0000 7f456fe00640 1 -- 172.21.15.103:0/412264614 <== osd.7 v2:172.21.15.136:6816/2248125128 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f455c0af670 con 0x7f455c0acba0 2024-09-16T02:04:17.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.682+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 --> [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f457806a850 con 0x7f455c0acba0 2024-09-16T02:04:17.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.683+0000 7f456fe00640 1 -- 172.21.15.103:0/412264614 <== osd.7 v2:172.21.15.136:6816/2248125128 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f457806a850 con 0x7f455c0acba0 2024-09-16T02:04:17.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.683+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 >> [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] conn(0x7f455c0acba0 msgr2=0x7f455c0aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.683+0000 7f457ce00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] conn(0x7f455c0acba0 0x7f455c0aefc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.684+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f455c0a3350 msgr2=0x7f455c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.684+0000 7f457ce00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f455c0a3350 0x7f455c0a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4568009bd0 tx=0x7f4568005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:17.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.684+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45781012a0 msgr2=0x7f45781a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:17.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.684+0000 7f457ce00640 1 --2- 172.21.15.103:0/412264614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45781012a0 0x7f45781a4b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f456400e9f0 tx=0x7f456400eeb0 comp rx=0 tx=0).stop 2024-09-16T02:04:17.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.684+0000 7f4577e00640 1 -- 172.21.15.103:0/412264614 reap_dead start 2024-09-16T02:04:17.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.684+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 shutdown_connections 2024-09-16T02:04:17.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.684+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 >> 172.21.15.103:0/412264614 conn(0x7f45780fc610 msgr2=0x7f4578103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:17.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.684+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 shutdown_connections 2024-09-16T02:04:17.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.684+0000 7f457ce00640 1 -- 172.21.15.103:0/412264614 wait complete. 2024-09-16T02:04:17.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:17 smithi103 ceph-mon[30398]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T02:04:18.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:17 smithi136 ceph-mon[35323]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T02:04:18.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:17 smithi172 ceph-mon[35340]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T02:04:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.885+0000 7f3eae800640 1 -- 172.21.15.103:0/814366695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ea8074620 msgr2=0x7f3ea8074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.885+0000 7f3eae800640 1 --2- 172.21.15.103:0/814366695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ea8074620 0x7f3ea8074a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009900 tx=0x7f3e9802a480 comp rx=0 tx=0).stop 2024-09-16T02:04:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.885+0000 7f3eae800640 1 -- 172.21.15.103:0/814366695 shutdown_connections 2024-09-16T02:04:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.885+0000 7f3eae800640 1 --2- 172.21.15.103:0/814366695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea810e3c0 0x7f3ea8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.885+0000 7f3eae800640 1 --2- 172.21.15.103:0/814366695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea8074f40 0x7f3ea810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.885+0000 7f3eae800640 1 --2- 172.21.15.103:0/814366695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ea8074620 0x7f3ea8074a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.885+0000 7f3eae800640 1 -- 172.21.15.103:0/814366695 >> 172.21.15.103:0/814366695 conn(0x7f3ea80fd8c0 msgr2=0x7f3ea80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.885+0000 7f3eae800640 1 -- 172.21.15.103:0/814366695 shutdown_connections 2024-09-16T02:04:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.886+0000 7f3eae800640 1 -- 172.21.15.103:0/814366695 wait complete. 2024-09-16T02:04:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.886+0000 7f3eae800640 1 Processor -- start 2024-09-16T02:04:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.886+0000 7f3eae800640 1 -- start start 2024-09-16T02:04:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3eae800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ea8074620 0x7f3ea819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3eae800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea8074f40 0x7f3ea819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3eae800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea810e3c0 0x7f3ea819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:18.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3eae800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea8114170 con 0x7f3ea8074f40 2024-09-16T02:04:18.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3eae800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea8113ff0 con 0x7f3ea810e3c0 2024-09-16T02:04:18.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3eae800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81142f0 con 0x7f3ea8074620 2024-09-16T02:04:18.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3ea6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea8074f40 0x7f3ea819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:18.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3ea6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea8074f40 0x7f3ea819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51676/0 (socket says 172.21.15.103:51676) 2024-09-16T02:04:18.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3ea6a00640 1 -- 172.21.15.103:0/1724131655 learned_addr learned my addr 172.21.15.103:0/1724131655 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:18.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3ea7400640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ea8074620 0x7f3ea819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:18.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3ea6a00640 1 -- 172.21.15.103:0/1724131655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ea8074620 msgr2=0x7f3ea819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:18.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3ea6a00640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ea8074620 0x7f3ea819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:18.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3ea6a00640 1 -- 172.21.15.103:0/1724131655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea810e3c0 msgr2=0x7f3ea819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:18.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.887+0000 7f3ea6a00640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea810e3c0 0x7f3ea819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:18.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.888+0000 7f3ea6a00640 1 -- 172.21.15.103:0/1724131655 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea819f700 con 0x7f3ea8074f40 2024-09-16T02:04:18.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.888+0000 7f3ea6a00640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea8074f40 0x7f3ea819ac90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400e9f0 tx=0x7f3e9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:18.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.888+0000 7f3e9fe00640 1 -- 172.21.15.103:0/1724131655 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9400cd80 con 0x7f3ea8074f40 2024-09-16T02:04:18.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.888+0000 7f3e9fe00640 1 -- 172.21.15.103:0/1724131655 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e94005500 con 0x7f3ea8074f40 2024-09-16T02:04:18.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.888+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea819f9f0 con 0x7f3ea8074f40 2024-09-16T02:04:18.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.888+0000 7f3e9fe00640 1 -- 172.21.15.103:0/1724131655 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94010690 con 0x7f3ea8074f40 2024-09-16T02:04:18.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.888+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea81a72d0 con 0x7f3ea8074f40 2024-09-16T02:04:18.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.889+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ea8062900 con 0x7f3ea8074f40 2024-09-16T02:04:18.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.892+0000 7f3e9fe00640 1 -- 172.21.15.103:0/1724131655 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3e940046c0 con 0x7f3ea8074f40 2024-09-16T02:04:18.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.893+0000 7f3e9fe00640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3e8c0a32d0 0x7f3e8c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:18.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.893+0000 7f3e9fe00640 1 -- 172.21.15.103:0/1724131655 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f3e940d02c0 con 0x7f3ea8074f40 2024-09-16T02:04:18.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.894+0000 7f3ea7400640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3e8c0a32d0 0x7f3e8c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:18.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.894+0000 7f3e9fe00640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] conn(0x7f3e8c0acb20 0x7f3e8c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:18.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.894+0000 7f3e9fe00640 1 -- 172.21.15.103:0/1724131655 --> [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3e94004470 con 0x7f3e8c0acb20 2024-09-16T02:04:18.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.894+0000 7f3e9fe00640 1 -- 172.21.15.103:0/1724131655 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f3e940d07a0 con 0x7f3ea8074f40 2024-09-16T02:04:18.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.894+0000 7f3ea7e00640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] conn(0x7f3e8c0acb20 0x7f3e8c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:18.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.894+0000 7f3ea7400640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3e8c0a32d0 0x7f3e8c0a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3e98005ab0 tx=0x7f3e98005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:18.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.895+0000 7f3ea7e00640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] conn(0x7f3e8c0acb20 0x7f3e8c0aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:18.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.895+0000 7f3e9fe00640 1 -- 172.21.15.103:0/1724131655 <== osd.1 v2:172.21.15.136:6800/1430190507 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3e94004470 con 0x7f3e8c0acb20 2024-09-16T02:04:18.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.909+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 --> [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ea8062d80 con 0x7f3e8c0acb20 2024-09-16T02:04:18.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.910+0000 7f3e9fe00640 1 -- 172.21.15.103:0/1724131655 <== osd.1 v2:172.21.15.136:6800/1430190507 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3ea8062d80 con 0x7f3e8c0acb20 2024-09-16T02:04:18.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 >> [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] conn(0x7f3e8c0acb20 msgr2=0x7f3e8c0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:18.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3eae800640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] conn(0x7f3e8c0acb20 0x7f3e8c0aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:18.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3e8c0a32d0 msgr2=0x7f3e8c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:18.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3eae800640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3e8c0a32d0 0x7f3e8c0a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3e98005ab0 tx=0x7f3e98005a20 comp rx=0 tx=0).stop 2024-09-16T02:04:18.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea8074f40 msgr2=0x7f3ea819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:18.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3eae800640 1 --2- 172.21.15.103:0/1724131655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea8074f40 0x7f3ea819ac90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400e9f0 tx=0x7f3e9400edf0 comp rx=0 tx=0).stop 2024-09-16T02:04:18.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3ea7e00640 1 -- 172.21.15.103:0/1724131655 reap_dead start 2024-09-16T02:04:18.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 shutdown_connections 2024-09-16T02:04:18.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 >> 172.21.15.103:0/1724131655 conn(0x7f3ea80fd8c0 msgr2=0x7f3ea80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:18.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 shutdown_connections 2024-09-16T02:04:18.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:17.911+0000 7f3eae800640 1 -- 172.21.15.103:0/1724131655 wait complete. 2024-09-16T02:04:19.081 INFO:teuthology.orchestra.run.smithi103.stdout:51539607626 2024-09-16T02:04:19.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T02:04:19.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.273+0000 7f25c1e00640 1 -- 172.21.15.103:0/407221806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc074f40 msgr2=0x7f25bc10dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.273+0000 7f25c1e00640 1 --2- 172.21.15.103:0/407221806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc074f40 0x7f25bc10dcd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009900 tx=0x7f25a802a480 comp rx=0 tx=0).stop 2024-09-16T02:04:19.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.274+0000 7f25c1e00640 1 -- 172.21.15.103:0/407221806 shutdown_connections 2024-09-16T02:04:19.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.274+0000 7f25c1e00640 1 --2- 172.21.15.103:0/407221806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25bc10e210 0x7f25bc110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.274+0000 7f25c1e00640 1 --2- 172.21.15.103:0/407221806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc074f40 0x7f25bc10dcd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.274+0000 7f25c1e00640 1 --2- 172.21.15.103:0/407221806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25bc074620 0x7f25bc074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.274+0000 7f25c1e00640 1 -- 172.21.15.103:0/407221806 >> 172.21.15.103:0/407221806 conn(0x7f25bc0fd710 msgr2=0x7f25bc0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:19.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.274+0000 7f25c1e00640 1 -- 172.21.15.103:0/407221806 shutdown_connections 2024-09-16T02:04:19.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.274+0000 7f25c1e00640 1 -- 172.21.15.103:0/407221806 wait complete. 2024-09-16T02:04:19.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.275+0000 7f25c1e00640 1 Processor -- start 2024-09-16T02:04:19.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.275+0000 7f25c1e00640 1 -- start start 2024-09-16T02:04:19.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25c1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25bc074620 0x7f25bc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25c1e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc074f40 0x7f25bc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25c1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25bc10e210 0x7f25bc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25c1e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc113fc0 con 0x7f25bc074f40 2024-09-16T02:04:19.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25c1e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc113e40 con 0x7f25bc074620 2024-09-16T02:04:19.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25c1e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc114140 con 0x7f25bc10e210 2024-09-16T02:04:19.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25bbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc074f40 0x7f25bc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25bbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc074f40 0x7f25bc19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52558/0 (socket says 172.21.15.103:52558) 2024-09-16T02:04:19.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25bbe00640 1 -- 172.21.15.103:0/224635652 learned_addr learned my addr 172.21.15.103:0/224635652 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:19.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25c1400640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25bc10e210 0x7f25bc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.276+0000 7f25c0a00640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25bc074620 0x7f25bc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25bbe00640 1 -- 172.21.15.103:0/224635652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25bc10e210 msgr2=0x7f25bc19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25bbe00640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25bc10e210 0x7f25bc19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25bbe00640 1 -- 172.21.15.103:0/224635652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25bc074620 msgr2=0x7f25bc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25bbe00640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25bc074620 0x7f25bc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25bbe00640 1 -- 172.21.15.103:0/224635652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc19f470 con 0x7f25bc074f40 2024-09-16T02:04:19.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25bbe00640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc074f40 0x7f25bc19aa00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f25a8006010 tx=0x7f25a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:19.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25b9600640 1 -- 172.21.15.103:0/224635652 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a803f030 con 0x7f25bc074f40 2024-09-16T02:04:19.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25b9600640 1 -- 172.21.15.103:0/224635652 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25a8003ce0 con 0x7f25bc074f40 2024-09-16T02:04:19.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25b9600640 1 -- 172.21.15.103:0/224635652 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8035070 con 0x7f25bc074f40 2024-09-16T02:04:19.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25bc19f700 con 0x7f25bc074f40 2024-09-16T02:04:19.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.277+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25bc19fb90 con 0x7f25bc074f40 2024-09-16T02:04:19.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.278+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f25bc062980 con 0x7f25bc074f40 2024-09-16T02:04:19.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.280+0000 7f25b9600640 1 -- 172.21.15.103:0/224635652 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f25a8042030 con 0x7f25bc074f40 2024-09-16T02:04:19.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.282+0000 7f25b9600640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f25a40a32d0 0x7f25a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.282+0000 7f25b9600640 1 -- 172.21.15.103:0/224635652 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f25a80f06e0 con 0x7f25bc074f40 2024-09-16T02:04:19.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.282+0000 7f25c0a00640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f25a40a32d0 0x7f25a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.282+0000 7f25b9600640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] conn(0x7f25a40acb20 0x7f25a40aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.282+0000 7f25b9600640 1 -- 172.21.15.103:0/224635652 --> [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25a803be50 con 0x7f25a40acb20 2024-09-16T02:04:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.282+0000 7f25b9600640 1 -- 172.21.15.103:0/224635652 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f25a80f0a90 con 0x7f25bc074f40 2024-09-16T02:04:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.283+0000 7f25c1400640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] conn(0x7f25a40acb20 0x7f25a40aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.283+0000 7f25c0a00640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f25a40a32d0 0x7f25a40a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25b000bbe0 tx=0x7f25b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.283+0000 7f25c1400640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] conn(0x7f25a40acb20 0x7f25a40aef40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.285+0000 7f25b9600640 1 -- 172.21.15.103:0/224635652 <== osd.3 v2:172.21.15.103:6810/2129636883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25a803be50 con 0x7f25a40acb20 2024-09-16T02:04:19.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.299+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 --> [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f25bc062e00 con 0x7f25a40acb20 2024-09-16T02:04:19.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.300+0000 7f25b9600640 1 -- 172.21.15.103:0/224635652 <== osd.3 v2:172.21.15.103:6810/2129636883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f25bc062e00 con 0x7f25a40acb20 2024-09-16T02:04:19.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.300+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] conn(0x7f25a40acb20 msgr2=0x7f25a40aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.300+0000 7f25c1e00640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] conn(0x7f25a40acb20 0x7f25a40aef40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.301+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f25a40a32d0 msgr2=0x7f25a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.301+0000 7f25c1e00640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f25a40a32d0 0x7f25a40a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25b000bbe0 tx=0x7f25b000b390 comp rx=0 tx=0).stop 2024-09-16T02:04:19.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.301+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc074f40 msgr2=0x7f25bc19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:19.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.301+0000 7f25c1e00640 1 --2- 172.21.15.103:0/224635652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc074f40 0x7f25bc19aa00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f25a8006010 tx=0x7f25a80042f0 comp rx=0 tx=0).stop 2024-09-16T02:04:19.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.301+0000 7f25c1400640 1 -- 172.21.15.103:0/224635652 reap_dead start 2024-09-16T02:04:19.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.301+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 shutdown_connections 2024-09-16T02:04:19.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.301+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 >> 172.21.15.103:0/224635652 conn(0x7f25bc0fd710 msgr2=0x7f25bc0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:19.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.301+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 shutdown_connections 2024-09-16T02:04:19.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:19.301+0000 7f25c1e00640 1 -- 172.21.15.103:0/224635652 wait complete. 2024-09-16T02:04:20.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:19 smithi103 ceph-mon[30398]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-16T02:04:20.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:19 smithi136 ceph-mon[35323]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-16T02:04:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:19 smithi172 ceph-mon[35340]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-16T02:04:20.636 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:20.638 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:20.666 INFO:teuthology.orchestra.run.smithi103.stdout:197568495658 2024-09-16T02:04:20.666 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T02:04:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:21.213 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:21.215 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:21.257 INFO:teuthology.orchestra.run.smithi103.stdout:94489280576 2024-09-16T02:04:21.258 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T02:04:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.479+0000 7f5924a00640 1 -- 172.21.15.103:0/3972259428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5920105d90 msgr2=0x7f5920106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.479+0000 7f5924a00640 1 --2- 172.21.15.103:0/3972259428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5920105d90 0x7f5920106220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5918009ca0 tx=0x7f591802a940 comp rx=0 tx=0).stop 2024-09-16T02:04:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.479+0000 7f5924a00640 1 -- 172.21.15.103:0/3972259428 shutdown_connections 2024-09-16T02:04:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.479+0000 7f5924a00640 1 --2- 172.21.15.103:0/3972259428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5920105d90 0x7f5920106220 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.479+0000 7f5924a00640 1 --2- 172.21.15.103:0/3972259428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59201012a0 0x7f5920105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.479+0000 7f5924a00640 1 --2- 172.21.15.103:0/3972259428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5920100980 0x7f5920100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.479+0000 7f5924a00640 1 -- 172.21.15.103:0/3972259428 >> 172.21.15.103:0/3972259428 conn(0x7f59200fc610 msgr2=0x7f59200fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.479+0000 7f5924a00640 1 -- 172.21.15.103:0/3972259428 shutdown_connections 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.479+0000 7f5924a00640 1 -- 172.21.15.103:0/3972259428 wait complete. 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.480+0000 7f5924a00640 1 Processor -- start 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.480+0000 7f5924a00640 1 -- start start 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.480+0000 7f5924a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5920100980 0x7f59201a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.480+0000 7f5924a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59201012a0 0x7f59201a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.480+0000 7f5924a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5920105d90 0x7f592019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.480+0000 7f5924a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59201183b0 con 0x7f5920105d90 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.480+0000 7f5924a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5920118230 con 0x7f5920100980 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.480+0000 7f5924a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5920118530 con 0x7f59201012a0 2024-09-16T02:04:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f5917e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59201012a0 0x7f59201a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f591f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5920105d90 0x7f592019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f5917e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59201012a0 0x7f59201a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:54890/0 (socket says 172.21.15.103:54890) 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f5917e00640 1 -- 172.21.15.103:0/3584971470 learned_addr learned my addr 172.21.15.103:0/3584971470 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f591e800640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5920100980 0x7f59201a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f591f200640 1 -- 172.21.15.103:0/3584971470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59201012a0 msgr2=0x7f59201a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f591f200640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59201012a0 0x7f59201a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f591f200640 1 -- 172.21.15.103:0/3584971470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5920100980 msgr2=0x7f59201a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f591f200640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5920100980 0x7f59201a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f591f200640 1 -- 172.21.15.103:0/3584971470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592019eeb0 con 0x7f5920105d90 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.481+0000 7f591f200640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5920105d90 0x7f592019e640 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5918006010 tx=0x7f591802f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.482+0000 7f5917400640 1 -- 172.21.15.103:0/3584971470 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918034040 con 0x7f5920105d90 2024-09-16T02:04:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.482+0000 7f5917400640 1 -- 172.21.15.103:0/3584971470 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5918003e50 con 0x7f5920105d90 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.482+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592019f140 con 0x7f5920105d90 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.482+0000 7f5917400640 1 -- 172.21.15.103:0/3584971470 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59180046a0 con 0x7f5920105d90 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.482+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59201ab320 con 0x7f5920105d90 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.483+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5920105180 con 0x7f5920105d90 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.485+0000 7f5917400640 1 -- 172.21.15.103:0/3584971470 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5918037020 con 0x7f5920105d90 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.486+0000 7f5917400640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f59040a3350 0x7f59040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.486+0000 7f5917400640 1 -- 172.21.15.103:0/3584971470 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f59180f0c10 con 0x7f5920105d90 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.486+0000 7f591e800640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f59040a3350 0x7f59040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.486+0000 7f5917400640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] conn(0x7f59040acba0 0x7f59040aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.486+0000 7f5917400640 1 -- 172.21.15.103:0/3584971470 --> [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59040af670 con 0x7f59040acba0 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.486+0000 7f5917400640 1 -- 172.21.15.103:0/3584971470 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f59180f0fc0 con 0x7f5920105d90 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.487+0000 7f591e800640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f59040a3350 0x7f59040a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5910009bd0 tx=0x7f5910005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.487+0000 7f5917e00640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] conn(0x7f59040acba0 0x7f59040aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.487+0000 7f5917e00640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] conn(0x7f59040acba0 0x7f59040aefc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.488+0000 7f5917400640 1 -- 172.21.15.103:0/3584971470 <== osd.5 v2:172.21.15.172:6808/1913335537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59040af670 con 0x7f59040acba0 2024-09-16T02:04:21.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.502+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 --> [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f592006b070 con 0x7f59040acba0 2024-09-16T02:04:21.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.503+0000 7f5917400640 1 -- 172.21.15.103:0/3584971470 <== osd.5 v2:172.21.15.172:6808/1913335537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f592006b070 con 0x7f59040acba0 2024-09-16T02:04:21.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 >> [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] conn(0x7f59040acba0 msgr2=0x7f59040aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f5924a00640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] conn(0x7f59040acba0 0x7f59040aefc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f59040a3350 msgr2=0x7f59040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f5924a00640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f59040a3350 0x7f59040a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5910009bd0 tx=0x7f5910005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:21.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5920105d90 msgr2=0x7f592019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f5924a00640 1 --2- 172.21.15.103:0/3584971470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5920105d90 0x7f592019e640 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5918006010 tx=0x7f591802f640 comp rx=0 tx=0).stop 2024-09-16T02:04:21.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f591f200640 1 -- 172.21.15.103:0/3584971470 reap_dead start 2024-09-16T02:04:21.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 shutdown_connections 2024-09-16T02:04:21.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 >> 172.21.15.103:0/3584971470 conn(0x7f59200fc610 msgr2=0x7f5920103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:21.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 shutdown_connections 2024-09-16T02:04:21.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:20.504+0000 7f5924a00640 1 -- 172.21.15.103:0/3584971470 wait complete. 2024-09-16T02:04:21.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:21 smithi103 ceph-mon[30398]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:04:21.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.904+0000 7f0193200640 1 -- 172.21.15.103:0/136954493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0194102b90 msgr2=0x7f0194102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.904+0000 7f0193200640 1 --2- 172.21.15.103:0/136954493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0194102b90 0x7f0194102f70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0184009900 tx=0x7f018402a480 comp rx=0 tx=0).stop 2024-09-16T02:04:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.904+0000 7f0193200640 1 -- 172.21.15.103:0/136954493 shutdown_connections 2024-09-16T02:04:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.904+0000 7f0193200640 1 --2- 172.21.15.103:0/136954493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f019410bfc0 0x7f019410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.904+0000 7f0193200640 1 --2- 172.21.15.103:0/136954493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01941034b0 0x7f019410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.904+0000 7f0193200640 1 --2- 172.21.15.103:0/136954493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0194102b90 0x7f0194102f70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.904+0000 7f0193200640 1 -- 172.21.15.103:0/136954493 >> 172.21.15.103:0/136954493 conn(0x7f01940fc610 msgr2=0x7f01940fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.904+0000 7f0193200640 1 -- 172.21.15.103:0/136954493 shutdown_connections 2024-09-16T02:04:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.904+0000 7f0193200640 1 -- 172.21.15.103:0/136954493 wait complete. 2024-09-16T02:04:21.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.905+0000 7f0193200640 1 Processor -- start 2024-09-16T02:04:21.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.905+0000 7f0193200640 1 -- start start 2024-09-16T02:04:21.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0193200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0194102b90 0x7f019419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0193200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01941034b0 0x7f019419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0193200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019410bfc0 0x7f019419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0193200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0194111e10 con 0x7f019410bfc0 2024-09-16T02:04:21.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0193200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0194111c90 con 0x7f0194102b90 2024-09-16T02:04:21.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0193200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0194111f90 con 0x7f01941034b0 2024-09-16T02:04:21.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0192800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019410bfc0 0x7f019419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0191e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0194102b90 0x7f019419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0192800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019410bfc0 0x7f019419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52594/0 (socket says 172.21.15.103:52594) 2024-09-16T02:04:21.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0192800640 1 -- 172.21.15.103:0/2626282494 learned_addr learned my addr 172.21.15.103:0/2626282494 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:21.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0191400640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01941034b0 0x7f019419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0192800640 1 -- 172.21.15.103:0/2626282494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01941034b0 msgr2=0x7f019419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0192800640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01941034b0 0x7f019419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0192800640 1 -- 172.21.15.103:0/2626282494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0194102b90 msgr2=0x7f019419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0192800640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0194102b90 0x7f019419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.906+0000 7f0192800640 1 -- 172.21.15.103:0/2626282494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019419f460 con 0x7f019410bfc0 2024-09-16T02:04:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.907+0000 7f0192800640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019410bfc0 0x7f019419ed80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f018c00eb10 tx=0x7f018c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.907+0000 7f018aa00640 1 -- 172.21.15.103:0/2626282494 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c004b20 con 0x7f019410bfc0 2024-09-16T02:04:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.907+0000 7f018aa00640 1 -- 172.21.15.103:0/2626282494 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018c004cc0 con 0x7f019410bfc0 2024-09-16T02:04:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.907+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019419f750 con 0x7f019410bfc0 2024-09-16T02:04:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.907+0000 7f018aa00640 1 -- 172.21.15.103:0/2626282494 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c010760 con 0x7f019410bfc0 2024-09-16T02:04:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.907+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01941a7030 con 0x7f019410bfc0 2024-09-16T02:04:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.908+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01940fec80 con 0x7f019410bfc0 2024-09-16T02:04:21.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.910+0000 7f018aa00640 1 -- 172.21.15.103:0/2626282494 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f018c005020 con 0x7f019410bfc0 2024-09-16T02:04:21.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.911+0000 7f018aa00640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f01780a3350 0x7f01780a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.911+0000 7f018aa00640 1 -- 172.21.15.103:0/2626282494 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f018c0d02a0 con 0x7f019410bfc0 2024-09-16T02:04:21.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.912+0000 7f0191e00640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f01780a3350 0x7f01780a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.912+0000 7f018aa00640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] conn(0x7f01780acba0 0x7f01780aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:21.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.912+0000 7f018aa00640 1 -- 172.21.15.103:0/2626282494 --> [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01780af670 con 0x7f01780acba0 2024-09-16T02:04:21.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.912+0000 7f018aa00640 1 -- 172.21.15.103:0/2626282494 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f018c0d0650 con 0x7f019410bfc0 2024-09-16T02:04:21.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.912+0000 7f0191400640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] conn(0x7f01780acba0 0x7f01780aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:21.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.912+0000 7f0191e00640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f01780a3350 0x7f01780a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0184000c00 tx=0x7f0184005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.913+0000 7f0191400640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] conn(0x7f01780acba0 0x7f01780aefc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:21.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.914+0000 7f018aa00640 1 -- 172.21.15.103:0/2626282494 <== osd.2 v2:172.21.15.172:6800/544248973 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01780af670 con 0x7f01780acba0 2024-09-16T02:04:21.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.928+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 --> [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f019406a850 con 0x7f01780acba0 2024-09-16T02:04:21.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.928+0000 7f018aa00640 1 -- 172.21.15.103:0/2626282494 <== osd.2 v2:172.21.15.172:6800/544248973 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f019406a850 con 0x7f01780acba0 2024-09-16T02:04:21.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 >> [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] conn(0x7f01780acba0 msgr2=0x7f01780aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0193200640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] conn(0x7f01780acba0 0x7f01780aefc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f01780a3350 msgr2=0x7f01780a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0193200640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f01780a3350 0x7f01780a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0184000c00 tx=0x7f0184005b50 comp rx=0 tx=0).stop 2024-09-16T02:04:21.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019410bfc0 msgr2=0x7f019419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:21.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0193200640 1 --2- 172.21.15.103:0/2626282494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019410bfc0 0x7f019419ed80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f018c00eb10 tx=0x7f018c00c2d0 comp rx=0 tx=0).stop 2024-09-16T02:04:21.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0192800640 1 -- 172.21.15.103:0/2626282494 reap_dead start 2024-09-16T02:04:21.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 shutdown_connections 2024-09-16T02:04:21.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 >> 172.21.15.103:0/2626282494 conn(0x7f01940fc610 msgr2=0x7f01940fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:21.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 shutdown_connections 2024-09-16T02:04:21.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:21.929+0000 7f0193200640 1 -- 172.21.15.103:0/2626282494 wait complete. 2024-09-16T02:04:22.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:21 smithi136 ceph-mon[35323]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:04:22.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:21 smithi172 ceph-mon[35340]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:04:22.669 INFO:teuthology.orchestra.run.smithi103.stdout:146028888116 2024-09-16T02:04:22.669 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T02:04:22.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.830+0000 7fb0c2000640 1 -- 172.21.15.103:0/1191016563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc109140 msgr2=0x7fb0bc110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.830+0000 7fb0c2000640 1 --2- 172.21.15.103:0/1191016563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc109140 0x7fb0bc110cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009ca0 tx=0x7fb0b402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.830+0000 7fb0c2000640 1 -- 172.21.15.103:0/1191016563 shutdown_connections 2024-09-16T02:04:22.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.830+0000 7fb0c2000640 1 --2- 172.21.15.103:0/1191016563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc109140 0x7fb0bc110cc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.830+0000 7fb0c2000640 1 --2- 172.21.15.103:0/1191016563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0bc1049c0 0x7fb0bc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.830+0000 7fb0c2000640 1 --2- 172.21.15.103:0/1191016563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc104010 0x7fb0bc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.830+0000 7fb0c2000640 1 -- 172.21.15.103:0/1191016563 >> 172.21.15.103:0/1191016563 conn(0x7fb0bc0ffa50 msgr2=0x7fb0bc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:22.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.830+0000 7fb0c2000640 1 -- 172.21.15.103:0/1191016563 shutdown_connections 2024-09-16T02:04:22.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.830+0000 7fb0c2000640 1 -- 172.21.15.103:0/1191016563 wait complete. 2024-09-16T02:04:22.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.831+0000 7fb0c2000640 1 Processor -- start 2024-09-16T02:04:22.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.832+0000 7fb0c2000640 1 -- start start 2024-09-16T02:04:22.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.832+0000 7fb0c2000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc104010 0x7fb0bc19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.832+0000 7fb0c2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0bc1049c0 0x7fb0bc19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.832+0000 7fb0c2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc109140 0x7fb0bc19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.832+0000 7fb0c2000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1141c0 con 0x7fb0bc109140 2024-09-16T02:04:22.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.832+0000 7fb0c2000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc114040 con 0x7fb0bc1049c0 2024-09-16T02:04:22.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.832+0000 7fb0c2000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc114340 con 0x7fb0bc104010 2024-09-16T02:04:22.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0bbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc109140 0x7fb0bc19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0bbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc109140 0x7fb0bc19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52618/0 (socket says 172.21.15.103:52618) 2024-09-16T02:04:22.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0bbe00640 1 -- 172.21.15.103:0/626215163 learned_addr learned my addr 172.21.15.103:0/626215163 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:22.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0bb400640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc104010 0x7fb0bc19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0baa00640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0bc1049c0 0x7fb0bc19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0bb400640 1 -- 172.21.15.103:0/626215163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0bc1049c0 msgr2=0x7fb0bc19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0bb400640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0bc1049c0 0x7fb0bc19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0bb400640 1 -- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc109140 msgr2=0x7fb0bc19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0bb400640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc109140 0x7fb0bc19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0bb400640 1 -- 172.21.15.103:0/626215163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc19f670 con 0x7fb0bc104010 2024-09-16T02:04:22.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0baa00640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0bc1049c0 0x7fb0bc19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:22.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.833+0000 7fb0bbe00640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc109140 0x7fb0bc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:22.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.834+0000 7fb0bb400640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc104010 0x7fb0bc19a6c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e9f0 tx=0x7fb0ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:22.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.834+0000 7fb0b3e00640 1 -- 172.21.15.103:0/626215163 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00ccc0 con 0x7fb0bc104010 2024-09-16T02:04:22.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.834+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc19f960 con 0x7fb0bc104010 2024-09-16T02:04:22.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.834+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc1a7240 con 0x7fb0bc104010 2024-09-16T02:04:22.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.835+0000 7fb0b3e00640 1 -- 172.21.15.103:0/626215163 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0ac005020 con 0x7fb0bc104010 2024-09-16T02:04:22.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.835+0000 7fb0b3e00640 1 -- 172.21.15.103:0/626215163 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac01c070 con 0x7fb0bc104010 2024-09-16T02:04:22.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.835+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb09c001030 con 0x7fb0bc104010 2024-09-16T02:04:22.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.839+0000 7fb0b3e00640 1 -- 172.21.15.103:0/626215163 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb0ac004020 con 0x7fb0bc104010 2024-09-16T02:04:22.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.840+0000 7fb0b3e00640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb0a00a33d0 0x7fb0a00a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.841+0000 7fb0b3e00640 1 -- 172.21.15.103:0/626215163 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fb0ac0cfc90 con 0x7fb0bc104010 2024-09-16T02:04:22.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.841+0000 7fb0baa00640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb0a00a33d0 0x7fb0a00a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.841+0000 7fb0b3e00640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] conn(0x7fb0a00acc20 0x7fb0a00af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:22.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.841+0000 7fb0b3e00640 1 -- 172.21.15.103:0/626215163 --> [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0ac005650 con 0x7fb0a00acc20 2024-09-16T02:04:22.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.841+0000 7fb0b3e00640 1 -- 172.21.15.103:0/626215163 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fb0ac0d0080 con 0x7fb0bc104010 2024-09-16T02:04:22.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.841+0000 7fb0bbe00640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] conn(0x7fb0a00acc20 0x7fb0a00af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:22.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.842+0000 7fb0baa00640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb0a00a33d0 0x7fb0a00a5890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0ffe70 tx=0x7fb0a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:22.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.842+0000 7fb0bbe00640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] conn(0x7fb0a00acc20 0x7fb0a00af040 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:22.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.843+0000 7fb0b3e00640 1 -- 172.21.15.103:0/626215163 <== osd.6 v2:172.21.15.103:6818/1754743602 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0ac005650 con 0x7fb0a00acc20 2024-09-16T02:04:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.857+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 --> [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb09c002dd0 con 0x7fb0a00acc20 2024-09-16T02:04:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.858+0000 7fb0b3e00640 1 -- 172.21.15.103:0/626215163 <== osd.6 v2:172.21.15.103:6818/1754743602 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb09c002dd0 con 0x7fb0a00acc20 2024-09-16T02:04:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.858+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] conn(0x7fb0a00acc20 msgr2=0x7fb0a00af040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.858+0000 7fb0c2000640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] conn(0x7fb0a00acc20 0x7fb0a00af040 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.858+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb0a00a33d0 msgr2=0x7fb0a00a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.858+0000 7fb0c2000640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb0a00a33d0 0x7fb0a00a5890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0ffe70 tx=0x7fb0a8005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:22.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.858+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc104010 msgr2=0x7fb0bc19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:22.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.858+0000 7fb0c2000640 1 --2- 172.21.15.103:0/626215163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc104010 0x7fb0bc19a6c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e9f0 tx=0x7fb0ac00eeb0 comp rx=0 tx=0).stop 2024-09-16T02:04:22.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.858+0000 7fb0bbe00640 1 -- 172.21.15.103:0/626215163 reap_dead start 2024-09-16T02:04:22.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.859+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 shutdown_connections 2024-09-16T02:04:22.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.859+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 >> 172.21.15.103:0/626215163 conn(0x7fb0bc0ffa50 msgr2=0x7fb0bc100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:22.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.859+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 shutdown_connections 2024-09-16T02:04:22.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:22.859+0000 7fb0c2000640 1 -- 172.21.15.103:0/626215163 wait complete. 2024-09-16T02:04:23.043 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:23.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.058+0000 7fd9e8800640 1 -- 172.21.15.103:0/3638173904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e0074eb0 msgr2=0x7fd9e0112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.058+0000 7fd9e8800640 1 --2- 172.21.15.103:0/3638173904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e0074eb0 0x7fd9e0112640 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009900 tx=0x7fd9d002a480 comp rx=0 tx=0).stop 2024-09-16T02:04:23.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.058+0000 7fd9e8800640 1 -- 172.21.15.103:0/3638173904 shutdown_connections 2024-09-16T02:04:23.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.058+0000 7fd9e8800640 1 --2- 172.21.15.103:0/3638173904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e0112b80 0x7fd9e0114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.058+0000 7fd9e8800640 1 --2- 172.21.15.103:0/3638173904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e0074eb0 0x7fd9e0112640 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.058+0000 7fd9e8800640 1 --2- 172.21.15.103:0/3638173904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9e0074590 0x7fd9e0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.058+0000 7fd9e8800640 1 -- 172.21.15.103:0/3638173904 >> 172.21.15.103:0/3638173904 conn(0x7fd9e00fd890 msgr2=0x7fd9e00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:23.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.058+0000 7fd9e8800640 1 -- 172.21.15.103:0/3638173904 shutdown_connections 2024-09-16T02:04:23.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.058+0000 7fd9e8800640 1 -- 172.21.15.103:0/3638173904 wait complete. 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.059+0000 7fd9e8800640 1 Processor -- start 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.059+0000 7fd9e8800640 1 -- start start 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.059+0000 7fd9e8800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e0074590 0x7fd9e01a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.059+0000 7fd9e8800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e0074eb0 0x7fd9e01a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.059+0000 7fd9e8800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9e0112b80 0x7fd9e019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.059+0000 7fd9e8800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01185a0 con 0x7fd9e0074eb0 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.059+0000 7fd9e8800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0118420 con 0x7fd9e0074590 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.059+0000 7fd9e8800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0118720 con 0x7fd9e0112b80 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e0074eb0 0x7fd9e01a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e0074eb0 0x7fd9e01a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52624/0 (socket says 172.21.15.103:52624) 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5400640 1 -- 172.21.15.103:0/3530988466 learned_addr learned my addr 172.21.15.103:0/3530988466 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:23.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5e00640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e0074590 0x7fd9e01a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e6800640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9e0112b80 0x7fd9e019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5400640 1 -- 172.21.15.103:0/3530988466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9e0112b80 msgr2=0x7fd9e019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5400640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9e0112b80 0x7fd9e019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5400640 1 -- 172.21.15.103:0/3530988466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e0074590 msgr2=0x7fd9e01a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5400640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e0074590 0x7fd9e01a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5400640 1 -- 172.21.15.103:0/3530988466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e019f040 con 0x7fd9e0074eb0 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5e00640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e0074590 0x7fd9e01a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.060+0000 7fd9e5400640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e0074eb0 0x7fd9e01a4c90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0004770 tx=0x7fd9d0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.061+0000 7fd9daa00640 1 -- 172.21.15.103:0/3530988466 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d003f070 con 0x7fd9e0074eb0 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.061+0000 7fd9daa00640 1 -- 172.21.15.103:0/3530988466 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d0031ca0 con 0x7fd9e0074eb0 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.061+0000 7fd9daa00640 1 -- 172.21.15.103:0/3530988466 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0035030 con 0x7fd9e0074eb0 2024-09-16T02:04:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.061+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e019f2d0 con 0x7fd9e0074eb0 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.061+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e01ab620 con 0x7fd9e0074eb0 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.062+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd9e0062980 con 0x7fd9e0074eb0 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.064+0000 7fd9daa00640 1 -- 172.21.15.103:0/3530988466 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd9d0042070 con 0x7fd9e0074eb0 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.065+0000 7fd9daa00640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd9c80a3350 0x7fd9c80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.065+0000 7fd9daa00640 1 -- 172.21.15.103:0/3530988466 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fd9d00f0610 con 0x7fd9e0074eb0 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.065+0000 7fd9e5e00640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd9c80a3350 0x7fd9c80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.066+0000 7fd9daa00640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] conn(0x7fd9c80acba0 0x7fd9c80aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.066+0000 7fd9daa00640 1 -- 172.21.15.103:0/3530988466 --> [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9d0038e90 con 0x7fd9c80acba0 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.066+0000 7fd9daa00640 1 -- 172.21.15.103:0/3530988466 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd9d00f09c0 con 0x7fd9e0074eb0 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.066+0000 7fd9e5e00640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd9c80a3350 0x7fd9c80a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd9e019fa50 tx=0x7fd9d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.066+0000 7fd9e6800640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] conn(0x7fd9c80acba0 0x7fd9c80aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:23.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.067+0000 7fd9e6800640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] conn(0x7fd9c80acba0 0x7fd9c80aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:23.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.067+0000 7fd9daa00640 1 -- 172.21.15.103:0/3530988466 <== osd.0 v2:172.21.15.103:6802/484703095 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9d0038e90 con 0x7fd9c80acba0 2024-09-16T02:04:23.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.083+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 --> [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd9e0062d80 con 0x7fd9c80acba0 2024-09-16T02:04:23.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.083+0000 7fd9daa00640 1 -- 172.21.15.103:0/3530988466 <== osd.0 v2:172.21.15.103:6802/484703095 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd9e0062d80 con 0x7fd9c80acba0 2024-09-16T02:04:23.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] conn(0x7fd9c80acba0 msgr2=0x7fd9c80aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e8800640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] conn(0x7fd9c80acba0 0x7fd9c80aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd9c80a3350 msgr2=0x7fd9c80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e8800640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd9c80a3350 0x7fd9c80a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd9e019fa50 tx=0x7fd9d4005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:23.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e0074eb0 msgr2=0x7fd9e01a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e8800640 1 --2- 172.21.15.103:0/3530988466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e0074eb0 0x7fd9e01a4c90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0004770 tx=0x7fd9d0003c10 comp rx=0 tx=0).stop 2024-09-16T02:04:23.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e6800640 1 -- 172.21.15.103:0/3530988466 reap_dead start 2024-09-16T02:04:23.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 shutdown_connections 2024-09-16T02:04:23.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 >> 172.21.15.103:0/3530988466 conn(0x7fd9e00fd890 msgr2=0x7fd9e00ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:23.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 shutdown_connections 2024-09-16T02:04:23.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.084+0000 7fd9e8800640 1 -- 172.21.15.103:0/3530988466 wait complete. 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.629+0000 7fc98da00640 1 -- 172.21.15.103:0/2941740031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9881047e0 msgr2=0x7fc988108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.629+0000 7fc98da00640 1 --2- 172.21.15.103:0/2941740031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9881047e0 0x7fc988108830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc974009900 tx=0x7fc97402a410 comp rx=0 tx=0).stop 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.629+0000 7fc98da00640 1 -- 172.21.15.103:0/2941740031 shutdown_connections 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.629+0000 7fc98da00640 1 --2- 172.21.15.103:0/2941740031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988108f60 0x7fc988110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.629+0000 7fc98da00640 1 --2- 172.21.15.103:0/2941740031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9881047e0 0x7fc988108830 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.629+0000 7fc98da00640 1 --2- 172.21.15.103:0/2941740031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc988103e30 0x7fc988104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.629+0000 7fc98da00640 1 -- 172.21.15.103:0/2941740031 >> 172.21.15.103:0/2941740031 conn(0x7fc9880ff910 msgr2=0x7fc988101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.629+0000 7fc98da00640 1 -- 172.21.15.103:0/2941740031 shutdown_connections 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.629+0000 7fc98da00640 1 -- 172.21.15.103:0/2941740031 wait complete. 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.630+0000 7fc98da00640 1 Processor -- start 2024-09-16T02:04:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.630+0000 7fc98da00640 1 -- start start 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc98da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988103e30 0x7fc98819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc98da00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9881047e0 0x7fc98819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc98da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc988108f60 0x7fc98819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc98da00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc988113fd0 con 0x7fc9881047e0 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc98da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc988113e50 con 0x7fc988108f60 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc98da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc988114150 con 0x7fc988103e30 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc987e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988103e30 0x7fc98819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc987e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988103e30 0x7fc98819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:54972/0 (socket says 172.21.15.103:54972) 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc987e00640 1 -- 172.21.15.103:0/1392739308 learned_addr learned my addr 172.21.15.103:0/1392739308 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc98d000640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc988108f60 0x7fc98819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.631+0000 7fc987400640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9881047e0 0x7fc98819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.632+0000 7fc98d000640 1 -- 172.21.15.103:0/1392739308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988103e30 msgr2=0x7fc98819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.632+0000 7fc98d000640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988103e30 0x7fc98819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.632+0000 7fc98d000640 1 -- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9881047e0 msgr2=0x7fc98819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.632+0000 7fc98d000640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9881047e0 0x7fc98819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.632+0000 7fc98d000640 1 -- 172.21.15.103:0/1392739308 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98819f440 con 0x7fc988108f60 2024-09-16T02:04:23.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.632+0000 7fc987e00640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988103e30 0x7fc98819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:23.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.632+0000 7fc98d000640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc988108f60 0x7fc98819ed60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc98000e940 tx=0x7fc98000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:23.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.632+0000 7fc984c00640 1 -- 172.21.15.103:0/1392739308 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980004d20 con 0x7fc988108f60 2024-09-16T02:04:23.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.632+0000 7fc984c00640 1 -- 172.21.15.103:0/1392739308 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc980005020 con 0x7fc988108f60 2024-09-16T02:04:23.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.632+0000 7fc984c00640 1 -- 172.21.15.103:0/1392739308 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9800116b0 con 0x7fc988108f60 2024-09-16T02:04:23.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.633+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98819f730 con 0x7fc988108f60 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.633+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9881a7010 con 0x7fc988108f60 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.634+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc988062980 con 0x7fc988108f60 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.635+0000 7fc984c00640 1 -- 172.21.15.103:0/1392739308 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc9800054d0 con 0x7fc988108f60 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.637+0000 7fc984c00640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc96c0a3350 0x7fc96c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.637+0000 7fc984c00640 1 -- 172.21.15.103:0/1392739308 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fc9800cf9a0 con 0x7fc988108f60 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.637+0000 7fc987e00640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc96c0a3350 0x7fc96c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.637+0000 7fc984c00640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] conn(0x7fc96c0acba0 0x7fc96c0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.637+0000 7fc984c00640 1 -- 172.21.15.103:0/1392739308 --> [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc980018ec0 con 0x7fc96c0acba0 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.637+0000 7fc984c00640 1 -- 172.21.15.103:0/1392739308 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fc9800d42a0 con 0x7fc988108f60 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.638+0000 7fc987400640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] conn(0x7fc96c0acba0 0x7fc96c0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.638+0000 7fc987e00640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc96c0a3350 0x7fc96c0a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc978011550 tx=0x7fc978005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.638+0000 7fc987400640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] conn(0x7fc96c0acba0 0x7fc96c0aefc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.639+0000 7fc984c00640 1 -- 172.21.15.103:0/1392739308 <== osd.9 v2:172.21.15.103:6826/1291384117 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc980018ec0 con 0x7fc96c0acba0 2024-09-16T02:04:23.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.653+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 --> [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc988062e00 con 0x7fc96c0acba0 2024-09-16T02:04:23.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc984c00640 1 -- 172.21.15.103:0/1392739308 <== osd.9 v2:172.21.15.103:6826/1291384117 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc988062e00 con 0x7fc96c0acba0 2024-09-16T02:04:23.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] conn(0x7fc96c0acba0 msgr2=0x7fc96c0aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98da00640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] conn(0x7fc96c0acba0 0x7fc96c0aefc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc96c0a3350 msgr2=0x7fc96c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98da00640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc96c0a3350 0x7fc96c0a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc978011550 tx=0x7fc978005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc988108f60 msgr2=0x7fc98819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98da00640 1 --2- 172.21.15.103:0/1392739308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc988108f60 0x7fc98819ed60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc98000e940 tx=0x7fc98000ee00 comp rx=0 tx=0).stop 2024-09-16T02:04:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98d000640 1 -- 172.21.15.103:0/1392739308 reap_dead start 2024-09-16T02:04:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 shutdown_connections 2024-09-16T02:04:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 >> 172.21.15.103:0/1392739308 conn(0x7fc9880ff910 msgr2=0x7fc988100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 shutdown_connections 2024-09-16T02:04:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.654+0000 7fc98da00640 1 -- 172.21.15.103:0/1392739308 wait complete. 2024-09-16T02:04:23.774 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:23 smithi103 ceph-mon[30398]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:04:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:23 smithi136 ceph-mon[35323]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:04:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:23 smithi172 ceph-mon[35340]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:04:24.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.799+0000 7f8ea1600640 1 -- 172.21.15.103:0/2604897229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e9c108f60 msgr2=0x7f8e9c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:24.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.799+0000 7f8ea1600640 1 --2- 172.21.15.103:0/2604897229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e9c108f60 0x7f8e9c110ae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009900 tx=0x7f8e9402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:04:24.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.799+0000 7f8ea1600640 1 -- 172.21.15.103:0/2604897229 shutdown_connections 2024-09-16T02:04:24.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.799+0000 7f8ea1600640 1 --2- 172.21.15.103:0/2604897229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e9c108f60 0x7f8e9c110ae0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:24.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.799+0000 7f8ea1600640 1 --2- 172.21.15.103:0/2604897229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e9c1047e0 0x7f8e9c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.799+0000 7f8ea1600640 1 --2- 172.21.15.103:0/2604897229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9c103e30 0x7f8e9c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.799+0000 7f8ea1600640 1 -- 172.21.15.103:0/2604897229 >> 172.21.15.103:0/2604897229 conn(0x7f8e9c0ff910 msgr2=0x7f8e9c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.799+0000 7f8ea1600640 1 -- 172.21.15.103:0/2604897229 shutdown_connections 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.799+0000 7f8ea1600640 1 -- 172.21.15.103:0/2604897229 wait complete. 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.800+0000 7f8ea1600640 1 Processor -- start 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.801+0000 7f8ea1600640 1 -- start start 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.801+0000 7f8ea1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e9c103e30 0x7f8e9c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.801+0000 7f8ea1600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9c1047e0 0x7f8e9c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.801+0000 7f8ea1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e9c108f60 0x7f8e9c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.801+0000 7f8ea1600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c113fc0 con 0x7f8e9c103e30 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.801+0000 7f8ea1600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c113e40 con 0x7f8e9c108f60 2024-09-16T02:04:24.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.801+0000 7f8ea1600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c114140 con 0x7f8e9c1047e0 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.801+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e9c103e30 0x7f8e9c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.801+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e9c103e30 0x7f8e9c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52670/0 (socket says 172.21.15.103:52670) 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.801+0000 7f8e9be00640 1 -- 172.21.15.103:0/3060206592 learned_addr learned my addr 172.21.15.103:0/3060206592 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e9b400640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9c1047e0 0x7f8e9c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8ea0c00640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e9c108f60 0x7f8e9c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e9be00640 1 -- 172.21.15.103:0/3060206592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9c1047e0 msgr2=0x7f8e9c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e9be00640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9c1047e0 0x7f8e9c19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e9be00640 1 -- 172.21.15.103:0/3060206592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e9c108f60 msgr2=0x7f8e9c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e9be00640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e9c108f60 0x7f8e9c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e9be00640 1 -- 172.21.15.103:0/3060206592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c19f3d0 con 0x7f8e9c103e30 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e9b400640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9c1047e0 0x7f8e9c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8ea0c00640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e9c108f60 0x7f8e9c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e9be00640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e9c103e30 0x7f8e9c19a420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00eb90 tx=0x7f8e8c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e98c00640 1 -- 172.21.15.103:0/3060206592 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c004d20 con 0x7f8e9c103e30 2024-09-16T02:04:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e98c00640 1 -- 172.21.15.103:0/3060206592 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8c00ee90 con 0x7f8e9c103e30 2024-09-16T02:04:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c19f6c0 con 0x7f8e9c103e30 2024-09-16T02:04:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8e98c00640 1 -- 172.21.15.103:0/3060206592 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c010830 con 0x7f8e9c103e30 2024-09-16T02:04:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.802+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1a6fa0 con 0x7f8e9c103e30 2024-09-16T02:04:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.803+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e9c062940 con 0x7f8e9c103e30 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.805+0000 7f8e98c00640 1 -- 172.21.15.103:0/3060206592 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8e8c005020 con 0x7f8e9c103e30 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.807+0000 7f8e98c00640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8e800a3350 0x7f8e800a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.807+0000 7f8e98c00640 1 -- 172.21.15.103:0/3060206592 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f8e8c0d0500 con 0x7f8e9c103e30 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.807+0000 7f8e9b400640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8e800a3350 0x7f8e800a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.807+0000 7f8e98c00640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] conn(0x7f8e800acba0 0x7f8e800aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.807+0000 7f8e98c00640 1 -- 172.21.15.103:0/3060206592 --> [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e8c005680 con 0x7f8e800acba0 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.807+0000 7f8e98c00640 1 -- 172.21.15.103:0/3060206592 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f8e8c0d08b0 con 0x7f8e9c103e30 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.807+0000 7f8ea0c00640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] conn(0x7f8e800acba0 0x7f8e800aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.808+0000 7f8e9b400640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8e800a3350 0x7f8e800a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0ffd30 tx=0x7f8e88005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.808+0000 7f8ea0c00640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] conn(0x7f8e800acba0 0x7f8e800aefc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.810+0000 7f8e98c00640 1 -- 172.21.15.103:0/3060206592 <== osd.8 v2:172.21.15.172:6816/1611593335 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e8c005680 con 0x7f8e800acba0 2024-09-16T02:04:24.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.824+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 --> [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e9c062dc0 con 0x7f8e800acba0 2024-09-16T02:04:24.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.825+0000 7f8e98c00640 1 -- 172.21.15.103:0/3060206592 <== osd.8 v2:172.21.15.172:6816/1611593335 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8e9c062dc0 con 0x7f8e800acba0 2024-09-16T02:04:24.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.825+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 >> [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] conn(0x7f8e800acba0 msgr2=0x7f8e800aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:24.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.825+0000 7f8ea1600640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] conn(0x7f8e800acba0 0x7f8e800aefc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:24.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.825+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8e800a3350 msgr2=0x7f8e800a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:24.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.825+0000 7f8ea1600640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f8e800a3350 0x7f8e800a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0ffd30 tx=0x7f8e88005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:24.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.825+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e9c103e30 msgr2=0x7f8e9c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:24.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.825+0000 7f8ea1600640 1 --2- 172.21.15.103:0/3060206592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e9c103e30 0x7f8e9c19a420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00eb90 tx=0x7f8e8c00c6a0 comp rx=0 tx=0).stop 2024-09-16T02:04:24.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.825+0000 7f8ea0c00640 1 -- 172.21.15.103:0/3060206592 reap_dead start 2024-09-16T02:04:24.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.825+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 shutdown_connections 2024-09-16T02:04:24.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.825+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 >> 172.21.15.103:0/3060206592 conn(0x7f8e9c0ff910 msgr2=0x7f8e9c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:24.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.826+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 shutdown_connections 2024-09-16T02:04:24.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:24.826+0000 7f8ea1600640 1 -- 172.21.15.103:0/3060206592 wait complete. 2024-09-16T02:04:25.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:25 smithi103 ceph-mon[30398]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T02:04:25.755 INFO:teuthology.orchestra.run.smithi103.stdout:176093659184 2024-09-16T02:04:25.756 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T02:04:26.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:25 smithi136 ceph-mon[35323]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T02:04:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:25 smithi172 ceph-mon[35340]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T02:04:26.478 INFO:teuthology.orchestra.run.smithi103.stdout:34359738447 2024-09-16T02:04:26.479 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T02:04:26.647 INFO:teuthology.orchestra.run.smithi103.stdout:274877906969 2024-09-16T02:04:26.648 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-16T02:04:26.952 INFO:teuthology.orchestra.run.smithi103.stdout:257698037792 2024-09-16T02:04:26.952 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-16T02:04:27.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.885+0000 7faadb200640 1 -- 172.21.15.103:0/2828795180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faadc1047e0 msgr2=0x7faadc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:27.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.885+0000 7faadb200640 1 --2- 172.21.15.103:0/2828795180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faadc1047e0 0x7faadc108830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faac4009900 tx=0x7faac402a410 comp rx=0 tx=0).stop 2024-09-16T02:04:27.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.885+0000 7faadb200640 1 -- 172.21.15.103:0/2828795180 shutdown_connections 2024-09-16T02:04:27.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.885+0000 7faadb200640 1 --2- 172.21.15.103:0/2828795180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faadc108f60 0x7faadc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.885+0000 7faadb200640 1 --2- 172.21.15.103:0/2828795180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faadc1047e0 0x7faadc108830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.885+0000 7faadb200640 1 --2- 172.21.15.103:0/2828795180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc103e30 0x7faadc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.885+0000 7faadb200640 1 -- 172.21.15.103:0/2828795180 >> 172.21.15.103:0/2828795180 conn(0x7faadc0ff910 msgr2=0x7faadc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.885+0000 7faadb200640 1 -- 172.21.15.103:0/2828795180 shutdown_connections 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.885+0000 7faadb200640 1 -- 172.21.15.103:0/2828795180 wait complete. 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.886+0000 7faadb200640 1 Processor -- start 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.886+0000 7faadb200640 1 -- start start 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.886+0000 7faadb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faadc103e30 0x7faadc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.886+0000 7faadb200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faadc1047e0 0x7faadc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.886+0000 7faadb200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc108f60 0x7faadc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.886+0000 7faadb200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faadc114010 con 0x7faadc1047e0 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.886+0000 7faadb200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faadc113e90 con 0x7faadc103e30 2024-09-16T02:04:27.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.886+0000 7faadb200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faadc114190 con 0x7faadc108f60 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faad3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faadc1047e0 0x7faadc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faad3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faadc1047e0 0x7faadc19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52652/0 (socket says 172.21.15.103:52652) 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faad3e00640 1 -- 172.21.15.103:0/206155144 learned_addr learned my addr 172.21.15.103:0/206155144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faad9e00640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faadc103e30 0x7faadc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faada800640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc108f60 0x7faadc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faad3e00640 1 -- 172.21.15.103:0/206155144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc108f60 msgr2=0x7faadc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faad3e00640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc108f60 0x7faadc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faad3e00640 1 -- 172.21.15.103:0/206155144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faadc103e30 msgr2=0x7faadc19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faad3e00640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faadc103e30 0x7faadc19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faad3e00640 1 -- 172.21.15.103:0/206155144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc19f460 con 0x7faadc1047e0 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faada800640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc108f60 0x7faadc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:27.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.887+0000 7faad3e00640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faadc1047e0 0x7faadc19a9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faac4004770 tx=0x7faac4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.888+0000 7faad2a00640 1 -- 172.21.15.103:0/206155144 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac403f070 con 0x7faadc1047e0 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.888+0000 7faad2a00640 1 -- 172.21.15.103:0/206155144 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac4031ca0 con 0x7faadc1047e0 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.888+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faadc19f6f0 con 0x7faadc1047e0 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.888+0000 7faad2a00640 1 -- 172.21.15.103:0/206155144 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4035030 con 0x7faadc1047e0 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.888+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faadc1a7030 con 0x7faadc1047e0 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.889+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faadc062980 con 0x7faadc1047e0 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.891+0000 7faad2a00640 1 -- 172.21.15.103:0/206155144 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7faac4042070 con 0x7faadc1047e0 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.892+0000 7faad2a00640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faac00a32d0 0x7faac00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.892+0000 7faad2a00640 1 -- 172.21.15.103:0/206155144 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7faac40f0520 con 0x7faadc1047e0 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.892+0000 7faad9e00640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faac00a32d0 0x7faac00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.892+0000 7faad2a00640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] conn(0x7faac00acb20 0x7faac00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.892+0000 7faad2a00640 1 -- 172.21.15.103:0/206155144 --> [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faac4038e90 con 0x7faac00acb20 2024-09-16T02:04:27.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.892+0000 7faad2a00640 1 -- 172.21.15.103:0/206155144 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7faac40f0a00 con 0x7faadc1047e0 2024-09-16T02:04:27.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.893+0000 7faada800640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] conn(0x7faac00acb20 0x7faac00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:27.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.893+0000 7faad9e00640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faac00a32d0 0x7faac00a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faacc009bd0 tx=0x7faacc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:27.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.893+0000 7faada800640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] conn(0x7faac00acb20 0x7faac00aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:27.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.894+0000 7faad2a00640 1 -- 172.21.15.103:0/206155144 <== osd.4 v2:172.21.15.136:6808/2555490296 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faac4038e90 con 0x7faac00acb20 2024-09-16T02:04:27.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.908+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 --> [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faadc062e00 con 0x7faac00acb20 2024-09-16T02:04:27.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.909+0000 7faad2a00640 1 -- 172.21.15.103:0/206155144 <== osd.4 v2:172.21.15.136:6808/2555490296 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faadc062e00 con 0x7faac00acb20 2024-09-16T02:04:27.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 >> [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] conn(0x7faac00acb20 msgr2=0x7faac00aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faadb200640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] conn(0x7faac00acb20 0x7faac00aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faac00a32d0 msgr2=0x7faac00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faadb200640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faac00a32d0 0x7faac00a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faacc009bd0 tx=0x7faacc005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faadc1047e0 msgr2=0x7faadc19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faadb200640 1 --2- 172.21.15.103:0/206155144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faadc1047e0 0x7faadc19a9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faac4004770 tx=0x7faac4003c10 comp rx=0 tx=0).stop 2024-09-16T02:04:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faada800640 1 -- 172.21.15.103:0/206155144 reap_dead start 2024-09-16T02:04:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 shutdown_connections 2024-09-16T02:04:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 >> 172.21.15.103:0/206155144 conn(0x7faadc0ff910 msgr2=0x7faadc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 shutdown_connections 2024-09-16T02:04:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:23.910+0000 7faadb200640 1 -- 172.21.15.103:0/206155144 wait complete. 2024-09-16T02:04:27.759 INFO:teuthology.orchestra.run.smithi103.stdout:124554051643 2024-09-16T02:04:27.759 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T02:04:27.989 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:27 smithi103 ceph-mon[30398]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:04:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:27 smithi136 ceph-mon[35323]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:04:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:27 smithi172 ceph-mon[35340]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:04:28.457 INFO:teuthology.orchestra.run.smithi103.stdout:68719476805 2024-09-16T02:04:28.458 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T02:04:28.783 INFO:teuthology.orchestra.run.smithi103.stdout:227633266727 2024-09-16T02:04:28.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-16T02:04:28.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-16T02:04:28.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 1]}]: dispatch 2024-09-16T02:04:28.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-16T02:04:28.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-16T02:04:28.876 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [10, 11]}]: dispatch 2024-09-16T02:04:28.876 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-16T02:04:28.876 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 7, 6]}]: dispatch 2024-09-16T02:04:28.876 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-16T02:04:28.876 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:28 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-16T02:04:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-16T02:04:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 1]}]: dispatch 2024-09-16T02:04:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-16T02:04:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-16T02:04:29.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [10, 11]}]: dispatch 2024-09-16T02:04:29.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-16T02:04:29.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 7, 6]}]: dispatch 2024-09-16T02:04:29.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-16T02:04:29.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:28 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-16T02:04:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-16T02:04:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 1]}]: dispatch 2024-09-16T02:04:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-16T02:04:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-16T02:04:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [10, 11]}]: dispatch 2024-09-16T02:04:29.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-16T02:04:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 7, 6]}]: dispatch 2024-09-16T02:04:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-16T02:04:29.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:28 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-16T02:04:29.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.150+0000 7f0f9e800640 1 -- 172.21.15.103:0/716452491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f98102b90 msgr2=0x7f0f98102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:29.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.150+0000 7f0f9e800640 1 --2- 172.21.15.103:0/716452491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f98102b90 0x7f0f98102f70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009900 tx=0x7f0f8c02a480 comp rx=0 tx=0).stop 2024-09-16T02:04:29.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.150+0000 7f0f9e800640 1 -- 172.21.15.103:0/716452491 shutdown_connections 2024-09-16T02:04:29.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.150+0000 7f0f9e800640 1 --2- 172.21.15.103:0/716452491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f9810bfc0 0x7f0f9810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:29.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.150+0000 7f0f9e800640 1 --2- 172.21.15.103:0/716452491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f981034b0 0x7f0f9810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:29.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.150+0000 7f0f9e800640 1 --2- 172.21.15.103:0/716452491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f98102b90 0x7f0f98102f70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:29.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.150+0000 7f0f9e800640 1 -- 172.21.15.103:0/716452491 >> 172.21.15.103:0/716452491 conn(0x7f0f980fc610 msgr2=0x7f0f980fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:29.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.150+0000 7f0f9e800640 1 -- 172.21.15.103:0/716452491 shutdown_connections 2024-09-16T02:04:29.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.150+0000 7f0f9e800640 1 -- 172.21.15.103:0/716452491 wait complete. 2024-09-16T02:04:29.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.151+0000 7f0f9e800640 1 Processor -- start 2024-09-16T02:04:29.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.151+0000 7f0f9e800640 1 -- start start 2024-09-16T02:04:29.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f98102b90 0x7f0f981982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:29.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f981034b0 0x7f0f981987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:29.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f9810bfc0 0x7f0f9819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9e800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0f98111dc0 con 0x7f0f9810bfc0 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9e800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0f98111c40 con 0x7f0f98102b90 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9e800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0f98111f40 con 0x7f0f981034b0 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f9810bfc0 0x7f0f9819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f98102b90 0x7f0f981982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f98102b90 0x7f0f981982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:36140/0 (socket says 172.21.15.103:36140) 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f9810bfc0 0x7f0f9819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41624/0 (socket says 172.21.15.103:41624) 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9d400640 1 -- 172.21.15.103:0/2701823181 learned_addr learned my addr 172.21.15.103:0/2701823181 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.152+0000 7f0f9ca00640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f981034b0 0x7f0f981987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.153+0000 7f0f9d400640 1 -- 172.21.15.103:0/2701823181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f981034b0 msgr2=0x7f0f981987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.153+0000 7f0f9d400640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f981034b0 0x7f0f981987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.153+0000 7f0f9d400640 1 -- 172.21.15.103:0/2701823181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f9810bfc0 msgr2=0x7f0f9819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:29.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.153+0000 7f0f9d400640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f9810bfc0 0x7f0f9819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:29.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.153+0000 7f0f9d400640 1 -- 172.21.15.103:0/2701823181 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9819d250 con 0x7f0f98102b90 2024-09-16T02:04:29.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.153+0000 7f0f9de00640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f9810bfc0 0x7f0f9819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:29.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.154+0000 7f0f9d400640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f98102b90 0x7f0f981982a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c006010 tx=0x7f0f8c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:29.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.154+0000 7f0f96000640 1 -- 172.21.15.103:0/2701823181 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c03f030 con 0x7f0f98102b90 2024-09-16T02:04:29.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.154+0000 7f0f96000640 1 -- 172.21.15.103:0/2701823181 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8c003ce0 con 0x7f0f98102b90 2024-09-16T02:04:29.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.154+0000 7f0f96000640 1 -- 172.21.15.103:0/2701823181 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c035070 con 0x7f0f98102b90 2024-09-16T02:04:29.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.154+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9819d4e0 con 0x7f0f98102b90 2024-09-16T02:04:29.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.155+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9819d970 con 0x7f0f98102b90 2024-09-16T02:04:29.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.156+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f78001030 con 0x7f0f98102b90 2024-09-16T02:04:29.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.160+0000 7f0f96000640 1 -- 172.21.15.103:0/2701823181 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0f8c042030 con 0x7f0f98102b90 2024-09-16T02:04:29.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.161+0000 7f0f96000640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0f800a3420 0x7f0f800a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.161+0000 7f0f96000640 1 -- 172.21.15.103:0/2701823181 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8248+0+0 (secure 0 0 0) 0x7f0f8c0f08c0 con 0x7f0f98102b90 2024-09-16T02:04:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.162+0000 7f0f9ca00640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0f800a3420 0x7f0f800a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.162+0000 7f0f96000640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] conn(0x7f0f800ad050 0x7f0f800af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.162+0000 7f0f96000640 1 -- 172.21.15.103:0/2701823181 --> [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f8c03be50 con 0x7f0f800ad050 2024-09-16T02:04:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.162+0000 7f0f96000640 1 -- 172.21.15.103:0/2701823181 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f0f8c0f0c70 con 0x7f0f98102b90 2024-09-16T02:04:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.162+0000 7f0f9de00640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] conn(0x7f0f800ad050 0x7f0f800af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.163+0000 7f0f9de00640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] conn(0x7f0f800ad050 0x7f0f800af4f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.163+0000 7f0f9ca00640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0f800a3420 0x7f0f800a58e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0f840058c0 tx=0x7f0f84005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.164+0000 7f0f96000640 1 -- 172.21.15.103:0/2701823181 <== osd.11 v2:172.21.15.172:6824/2746302927 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f8c03be50 con 0x7f0f800ad050 2024-09-16T02:04:29.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.180+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 --> [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f78002dd0 con 0x7f0f800ad050 2024-09-16T02:04:29.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.181+0000 7f0f96000640 1 -- 172.21.15.103:0/2701823181 <== osd.11 v2:172.21.15.172:6824/2746302927 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0f78002dd0 con 0x7f0f800ad050 2024-09-16T02:04:29.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.181+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 >> [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] conn(0x7f0f800ad050 msgr2=0x7f0f800af4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:29.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.182+0000 7f0f9e800640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] conn(0x7f0f800ad050 0x7f0f800af4f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:29.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.182+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0f800a3420 msgr2=0x7f0f800a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:29.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.182+0000 7f0f9e800640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0f800a3420 0x7f0f800a58e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0f840058c0 tx=0x7f0f84005850 comp rx=0 tx=0).stop 2024-09-16T02:04:29.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.182+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f98102b90 msgr2=0x7f0f981982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:29.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.182+0000 7f0f9e800640 1 --2- 172.21.15.103:0/2701823181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f98102b90 0x7f0f981982a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c006010 tx=0x7f0f8c0042f0 comp rx=0 tx=0).stop 2024-09-16T02:04:29.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.182+0000 7f0f9de00640 1 -- 172.21.15.103:0/2701823181 reap_dead start 2024-09-16T02:04:29.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.182+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 shutdown_connections 2024-09-16T02:04:29.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.182+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 >> 172.21.15.103:0/2701823181 conn(0x7f0f980fc610 msgr2=0x7f0f980fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:29.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.182+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 shutdown_connections 2024-09-16T02:04:29.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:29.183+0000 7f0f9e800640 1 -- 172.21.15.103:0/2701823181 wait complete. 2024-09-16T02:04:29.521 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:29.522 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:29.715 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:30.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:04:30.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-16T02:04:30.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 1]}]': finished 2024-09-16T02:04:30.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-16T02:04:30.050 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [10, 11]}]': finished 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 7, 6]}]': finished 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:29 smithi172 ceph-mon[35340]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 1]}]': finished 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-16T02:04:30.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [10, 11]}]': finished 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 7, 6]}]': finished 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:29 smithi136 ceph-mon[35323]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 1]}]': finished 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [10, 11]}]': finished 2024-09-16T02:04:30.052 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-16T02:04:30.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 7, 6]}]': finished 2024-09-16T02:04:30.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-16T02:04:30.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-16T02:04:30.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:29 smithi103 ceph-mon[30398]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T02:04:30.053 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:30.054 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:30.054 INFO:teuthology.orchestra.run.smithi103.stdout:304942678040 2024-09-16T02:04:30.055 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-16T02:04:30.102 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:30.108 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:30.108 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:30.221 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:30.652 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:30.658 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:30.892 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:30 smithi103 ceph-mon[30398]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T02:04:30.892 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:04:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:30 smithi136 ceph-mon[35323]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T02:04:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:04:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:30 smithi172 ceph-mon[35340]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T02:04:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:04:31.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.418+0000 7f65f8e00640 1 -- 172.21.15.103:0/2437215766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65f41079c0 msgr2=0x7f65f4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:31.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.418+0000 7f65f8e00640 1 --2- 172.21.15.103:0/2437215766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65f41079c0 0x7f65f4107da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009900 tx=0x7f65e402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.418+0000 7f65f8e00640 1 -- 172.21.15.103:0/2437215766 shutdown_connections 2024-09-16T02:04:31.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.418+0000 7f65f8e00640 1 --2- 172.21.15.103:0/2437215766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65f4102360 0x7f65f410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.418+0000 7f65f8e00640 1 --2- 172.21.15.103:0/2437215766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f41019c0 0x7f65f4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.418+0000 7f65f8e00640 1 --2- 172.21.15.103:0/2437215766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65f41079c0 0x7f65f4107da0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.418+0000 7f65f8e00640 1 -- 172.21.15.103:0/2437215766 >> 172.21.15.103:0/2437215766 conn(0x7f65f40fd6e0 msgr2=0x7f65f40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:31.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.418+0000 7f65f8e00640 1 -- 172.21.15.103:0/2437215766 shutdown_connections 2024-09-16T02:04:31.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.418+0000 7f65f8e00640 1 -- 172.21.15.103:0/2437215766 wait complete. 2024-09-16T02:04:31.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.419+0000 7f65f8e00640 1 Processor -- start 2024-09-16T02:04:31.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.419+0000 7f65f8e00640 1 -- start start 2024-09-16T02:04:31.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.419+0000 7f65f8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65f41019c0 0x7f65f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:31.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.419+0000 7f65f8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65f4102360 0x7f65f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:31.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.419+0000 7f65f8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f41079c0 0x7f65f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:31.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.419+0000 7f65f8e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4111dd0 con 0x7f65f41079c0 2024-09-16T02:04:31.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.419+0000 7f65f8e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4111c50 con 0x7f65f41019c0 2024-09-16T02:04:31.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.419+0000 7f65f8e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4111f50 con 0x7f65f4102360 2024-09-16T02:04:31.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65f3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65f41019c0 0x7f65f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:31.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65f3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65f41019c0 0x7f65f40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:36150/0 (socket says 172.21.15.103:36150) 2024-09-16T02:04:31.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65f3400640 1 -- 172.21.15.103:0/3898941604 learned_addr learned my addr 172.21.15.103:0/3898941604 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:31.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65f3e00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f41079c0 0x7f65f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:31.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65f3400640 1 -- 172.21.15.103:0/3898941604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65f4102360 msgr2=0x7f65f4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:31.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65eaa00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65f4102360 0x7f65f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:31.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65f3400640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65f4102360 0x7f65f4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65f3400640 1 -- 172.21.15.103:0/3898941604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f41079c0 msgr2=0x7f65f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:31.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65f3400640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f41079c0 0x7f65f4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65f3400640 1 -- 172.21.15.103:0/3898941604 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f41a6710 con 0x7f65f41019c0 2024-09-16T02:04:31.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65eaa00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65f4102360 0x7f65f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:31.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.420+0000 7f65f3400640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65f41019c0 0x7f65f40744f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65e402a9e0 tx=0x7f65e4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:31.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.421+0000 7f65f0c00640 1 -- 172.21.15.103:0/3898941604 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e403f070 con 0x7f65f41019c0 2024-09-16T02:04:31.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.421+0000 7f65f0c00640 1 -- 172.21.15.103:0/3898941604 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4031c30 con 0x7f65f41019c0 2024-09-16T02:04:31.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.421+0000 7f65f0c00640 1 -- 172.21.15.103:0/3898941604 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4035030 con 0x7f65f41019c0 2024-09-16T02:04:31.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.421+0000 7f65f8e00640 1 -- 172.21.15.103:0/3898941604 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f41a69a0 con 0x7f65f41019c0 2024-09-16T02:04:31.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.421+0000 7f65f8e00640 1 -- 172.21.15.103:0/3898941604 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f41a6ed0 con 0x7f65f41019c0 2024-09-16T02:04:31.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.422+0000 7f65f8e00640 1 -- 172.21.15.103:0/3898941604 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f4101e20 con 0x7f65f41019c0 2024-09-16T02:04:31.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.424+0000 7f65f0c00640 1 -- 172.21.15.103:0/3898941604 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f65e4042070 con 0x7f65f41019c0 2024-09-16T02:04:31.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.425+0000 7f65f0c00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65d80a3350 0x7f65d80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:31.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.425+0000 7f65f0c00640 1 -- 172.21.15.103:0/3898941604 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f65e40f0090 con 0x7f65f41019c0 2024-09-16T02:04:31.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.425+0000 7f65eaa00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65d80a3350 0x7f65d80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:31.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.426+0000 7f65eaa00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65d80a3350 0x7f65d80a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65f4075f80 tx=0x7f65dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:31.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.429+0000 7f65f0c00640 1 -- 172.21.15.103:0/3898941604 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65e40bcc40 con 0x7f65f41019c0 2024-09-16T02:04:31.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.612+0000 7f65f8e00640 1 -- 172.21.15.103:0/3898941604 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f65f41a7170 con 0x7f65f41019c0 2024-09-16T02:04:31.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.613+0000 7f65f0c00640 1 -- 172.21.15.103:0/3898941604 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f65e4031de0 con 0x7f65f41019c0 2024-09-16T02:04:31.615 INFO:teuthology.orchestra.run.smithi103.stdout:146028888118 2024-09-16T02:04:31.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.615+0000 7f65f8e00640 1 -- 172.21.15.103:0/3898941604 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65d80a3350 msgr2=0x7f65d80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.615+0000 7f65f8e00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65d80a3350 0x7f65d80a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65f4075f80 tx=0x7f65dc005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.615+0000 7f65f8e00640 1 -- 172.21.15.103:0/3898941604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65f41019c0 msgr2=0x7f65f40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.615+0000 7f65f8e00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65f41019c0 0x7f65f40744f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65e402a9e0 tx=0x7f65e4003c10 comp rx=0 tx=0).stop 2024-09-16T02:04:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.615+0000 7f65f8e00640 1 -- 172.21.15.103:0/3898941604 shutdown_connections 2024-09-16T02:04:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.615+0000 7f65f8e00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65d80a3350 0x7f65d80a5810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.615+0000 7f65f8e00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f41079c0 0x7f65f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.615+0000 7f65f8e00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65f4102360 0x7f65f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.615+0000 7f65f8e00640 1 --2- 172.21.15.103:0/3898941604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65f41019c0 0x7f65f40744f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.615+0000 7f65f8e00640 1 -- 172.21.15.103:0/3898941604 >> 172.21.15.103:0/3898941604 conn(0x7f65f40fd6e0 msgr2=0x7f65f40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.616+0000 7f65f8e00640 1 -- 172.21.15.103:0/3898941604 shutdown_connections 2024-09-16T02:04:31.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.616+0000 7f65f8e00640 1 -- 172.21.15.103:0/3898941604 wait complete. 2024-09-16T02:04:31.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.630+0000 7f43eba00640 1 -- 172.21.15.103:0/445694366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43e4104010 msgr2=0x7f43e41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:31.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.630+0000 7f43eba00640 1 --2- 172.21.15.103:0/445694366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43e4104010 0x7f43e41043f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009860 tx=0x7f43d402a340 comp rx=0 tx=0).stop 2024-09-16T02:04:31.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.630+0000 7f43eba00640 1 -- 172.21.15.103:0/445694366 shutdown_connections 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.630+0000 7f43eba00640 1 --2- 172.21.15.103:0/445694366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4109140 0x7f43e4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.630+0000 7f43eba00640 1 --2- 172.21.15.103:0/445694366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43e41049c0 0x7f43e4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.630+0000 7f43eba00640 1 --2- 172.21.15.103:0/445694366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43e4104010 0x7f43e41043f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.630+0000 7f43eba00640 1 -- 172.21.15.103:0/445694366 >> 172.21.15.103:0/445694366 conn(0x7f43e40ffa50 msgr2=0x7f43e4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.630+0000 7f43eba00640 1 -- 172.21.15.103:0/445694366 shutdown_connections 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.630+0000 7f43eba00640 1 -- 172.21.15.103:0/445694366 wait complete. 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.631+0000 7f43eba00640 1 Processor -- start 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.631+0000 7f43eba00640 1 -- start start 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.631+0000 7f43eba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4104010 0x7f43e419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.631+0000 7f43eba00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43e41049c0 0x7f43e419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.631+0000 7f43eba00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43e4109140 0x7f43e419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.631+0000 7f43eba00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4114140 con 0x7f43e4109140 2024-09-16T02:04:31.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.631+0000 7f43eba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4113fc0 con 0x7f43e4104010 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.631+0000 7f43eba00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41142c0 con 0x7f43e41049c0 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43dbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43e41049c0 0x7f43e419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43e9000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4104010 0x7f43e419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43e9a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43e4109140 0x7f43e419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43dbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43e41049c0 0x7f43e419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:48696/0 (socket says 172.21.15.103:48696) 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43dbe00640 1 -- 172.21.15.103:0/3196119525 learned_addr learned my addr 172.21.15.103:0/3196119525 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43e9000640 1 -- 172.21.15.103:0/3196119525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43e41049c0 msgr2=0x7f43e419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43e9000640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43e41049c0 0x7f43e419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43e9000640 1 -- 172.21.15.103:0/3196119525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43e4109140 msgr2=0x7f43e419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43e9000640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43e4109140 0x7f43e419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43e9000640 1 -- 172.21.15.103:0/3196119525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e419f5f0 con 0x7f43e4104010 2024-09-16T02:04:31.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.632+0000 7f43e9000640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4104010 0x7f43e419a640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f43d4035040 tx=0x7f43d4033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.633+0000 7f43e2000640 1 -- 172.21.15.103:0/3196119525 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d403f020 con 0x7f43e4104010 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.633+0000 7f43e2000640 1 -- 172.21.15.103:0/3196119525 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d4030550 con 0x7f43e4104010 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.633+0000 7f43e2000640 1 -- 172.21.15.103:0/3196119525 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4037040 con 0x7f43e4104010 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.633+0000 7f43eba00640 1 -- 172.21.15.103:0/3196119525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e419f880 con 0x7f43e4104010 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.633+0000 7f43eba00640 1 -- 172.21.15.103:0/3196119525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e41a71c0 con 0x7f43e4104010 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.634+0000 7f43eba00640 1 -- 172.21.15.103:0/3196119525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e41054f0 con 0x7f43e4104010 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.636+0000 7f43e2000640 1 -- 172.21.15.103:0/3196119525 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f43d4042020 con 0x7f43e4104010 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.637+0000 7f43e2000640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43c80a32d0 0x7f43c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.637+0000 7f43e2000640 1 -- 172.21.15.103:0/3196119525 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f43d40efa50 con 0x7f43e4104010 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.637+0000 7f43dbe00640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43c80a32d0 0x7f43c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:31.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.638+0000 7f43dbe00640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43c80a32d0 0x7f43c80a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f43e40ffe70 tx=0x7f43cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:31.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.641+0000 7f43e2000640 1 -- 172.21.15.103:0/3196119525 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43d4049020 con 0x7f43e4104010 2024-09-16T02:04:31.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:31 smithi103 ceph-mon[30398]: pgmap v310: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-16T02:04:31.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:31 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3898941604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T02:04:31.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.824+0000 7f43eba00640 1 -- 172.21.15.103:0/3196119525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f43e410c320 con 0x7f43e4104010 2024-09-16T02:04:31.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.824+0000 7f43e2000640 1 -- 172.21.15.103:0/3196119525 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f43d40bc6b0 con 0x7f43e4104010 2024-09-16T02:04:31.827 INFO:teuthology.orchestra.run.smithi103.stdout:257698037793 2024-09-16T02:04:31.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 -- 172.21.15.103:0/3196119525 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43c80a32d0 msgr2=0x7f43c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:31.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43c80a32d0 0x7f43c80a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f43e40ffe70 tx=0x7f43cc005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:31.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 -- 172.21.15.103:0/3196119525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4104010 msgr2=0x7f43e419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:31.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4104010 0x7f43e419a640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f43d4035040 tx=0x7f43d4033b60 comp rx=0 tx=0).stop 2024-09-16T02:04:31.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 -- 172.21.15.103:0/3196119525 shutdown_connections 2024-09-16T02:04:31.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43c80a32d0 0x7f43c80a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43e4109140 0x7f43e419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43e41049c0 0x7f43e419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 --2- 172.21.15.103:0/3196119525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4104010 0x7f43e419a640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:31.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 -- 172.21.15.103:0/3196119525 >> 172.21.15.103:0/3196119525 conn(0x7f43e40ffa50 msgr2=0x7f43e4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:31.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 -- 172.21.15.103:0/3196119525 shutdown_connections 2024-09-16T02:04:31.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:31.827+0000 7f43eba00640 1 -- 172.21.15.103:0/3196119525 wait complete. 2024-09-16T02:04:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:31 smithi136 ceph-mon[35323]: pgmap v310: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-16T02:04:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:31 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3898941604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T02:04:32.044 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:31 smithi172 ceph-mon[35340]: pgmap v310: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-16T02:04:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:31 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3898941604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T02:04:32.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.063+0000 7f617bc00640 1 -- 172.21.15.103:0/220866321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174074620 msgr2=0x7f6174074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.063+0000 7f617bc00640 1 --2- 172.21.15.103:0/220866321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174074620 0x7f6174074a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6168009860 tx=0x7f616802a3d0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.063+0000 7f617bc00640 1 -- 172.21.15.103:0/220866321 shutdown_connections 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.063+0000 7f617bc00640 1 --2- 172.21.15.103:0/220866321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617410e3c0 0x7f6174110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.063+0000 7f617bc00640 1 --2- 172.21.15.103:0/220866321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6174074f40 0x7f617410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.063+0000 7f617bc00640 1 --2- 172.21.15.103:0/220866321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174074620 0x7f6174074a00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.063+0000 7f617bc00640 1 -- 172.21.15.103:0/220866321 >> 172.21.15.103:0/220866321 conn(0x7f61740fd8c0 msgr2=0x7f61740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.063+0000 7f617bc00640 1 -- 172.21.15.103:0/220866321 shutdown_connections 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.063+0000 7f617bc00640 1 -- 172.21.15.103:0/220866321 wait complete. 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f617bc00640 1 Processor -- start 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f617bc00640 1 -- start start 2024-09-16T02:04:32.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f617bc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6174074620 0x7f617419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f617bc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174074f40 0x7f617419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f617bc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f617410e3c0 0x7f617419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f617bc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61741141c0 con 0x7f617410e3c0 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f617bc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6174114040 con 0x7f6174074f40 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f617bc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6174114340 con 0x7f6174074620 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f6179200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6174074620 0x7f617419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f6173e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174074f40 0x7f617419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f6179200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6174074620 0x7f617419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:48700/0 (socket says 172.21.15.103:48700) 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f6179200640 1 -- 172.21.15.103:0/1757430708 learned_addr learned my addr 172.21.15.103:0/1757430708 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.064+0000 7f6173e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174074f40 0x7f617419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:36190/0 (socket says 172.21.15.103:36190) 2024-09-16T02:04:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6179c00640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f617410e3c0 0x7f617419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6173e00640 1 -- 172.21.15.103:0/1757430708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6174074620 msgr2=0x7f617419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6173e00640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6174074620 0x7f617419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6173e00640 1 -- 172.21.15.103:0/1757430708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f617410e3c0 msgr2=0x7f617419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6173e00640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f617410e3c0 0x7f617419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6173e00640 1 -- 172.21.15.103:0/1757430708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617419f650 con 0x7f6174074f40 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6179200640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6174074620 0x7f617419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6173e00640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174074f40 0x7f617419abe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f616000e790 tx=0x7f616000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6171600640 1 -- 172.21.15.103:0/1757430708 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616000eea0 con 0x7f6174074f40 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f617bc00640 1 -- 172.21.15.103:0/1757430708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617419f940 con 0x7f6174074f40 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6171600640 1 -- 172.21.15.103:0/1757430708 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f616000ccc0 con 0x7f6174074f40 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.065+0000 7f6171600640 1 -- 172.21.15.103:0/1757430708 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160018780 con 0x7f6174074f40 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.066+0000 7f617bc00640 1 -- 172.21.15.103:0/1757430708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61741a7220 con 0x7f6174074f40 2024-09-16T02:04:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.067+0000 7f617bc00640 1 -- 172.21.15.103:0/1757430708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6174074a00 con 0x7f6174074f40 2024-09-16T02:04:32.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.069+0000 7f6171600640 1 -- 172.21.15.103:0/1757430708 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6160010430 con 0x7f6174074f40 2024-09-16T02:04:32.293 INFO:teuthology.orchestra.run.smithi103.stdout:51539607629 2024-09-16T02:04:32.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.071+0000 7f6171600640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f615c0a32d0 0x7f615c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:32.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.071+0000 7f6171600640 1 -- 172.21.15.103:0/1757430708 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f61600cfdf0 con 0x7f6174074f40 2024-09-16T02:04:32.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.071+0000 7f6179200640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f615c0a32d0 0x7f615c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:32.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.071+0000 7f6179200640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f615c0a32d0 0x7f615c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f61680095f0 tx=0x7f6168005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.072+0000 7f6171600640 1 -- 172.21.15.103:0/1757430708 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f616009c2f0 con 0x7f6174074f40 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.259+0000 7f617bc00640 1 -- 172.21.15.103:0/1757430708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f617402d7c0 con 0x7f6174074f40 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.260+0000 7f6171600640 1 -- 172.21.15.103:0/1757430708 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f61600a05f0 con 0x7f6174074f40 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 -- 172.21.15.103:0/1757430708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f615c0a32d0 msgr2=0x7f615c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f615c0a32d0 0x7f615c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f61680095f0 tx=0x7f6168005cb0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 -- 172.21.15.103:0/1757430708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174074f40 msgr2=0x7f617419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174074f40 0x7f617419abe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f616000e790 tx=0x7f616000ec50 comp rx=0 tx=0).stop 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 -- 172.21.15.103:0/1757430708 shutdown_connections 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f615c0a32d0 0x7f615c0a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f617410e3c0 0x7f617419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174074f40 0x7f617419abe0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 --2- 172.21.15.103:0/1757430708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6174074620 0x7f617419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 -- 172.21.15.103:0/1757430708 >> 172.21.15.103:0/1757430708 conn(0x7f61740fd8c0 msgr2=0x7f61740fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:32.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 -- 172.21.15.103:0/1757430708 shutdown_connections 2024-09-16T02:04:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:32.262+0000 7f617bc00640 1 -- 172.21.15.103:0/1757430708 wait complete. 2024-09-16T02:04:32.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:32 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3196119525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T02:04:32.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:32.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:32.883 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:32.883 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:32 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:32.883 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:32 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1757430708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T02:04:33.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:32 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3196119525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T02:04:33.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:33.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:33.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:33.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:32 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:33.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:32 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1757430708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T02:04:33.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:32 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3196119525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T02:04:33.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:33.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:33.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:33.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:32 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:33.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:32 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1757430708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T02:04:33.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.173+0000 7fc7cbe00640 1 -- 172.21.15.103:0/3230783337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c4102790 msgr2=0x7fc7c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:33.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.173+0000 7fc7cbe00640 1 --2- 172.21.15.103:0/3230783337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c4102790 0x7fc7c41067e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009900 tx=0x7fc7b002a3f0 comp rx=0 tx=0).stop 2024-09-16T02:04:33.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.173+0000 7fc7cbe00640 1 -- 172.21.15.103:0/3230783337 shutdown_connections 2024-09-16T02:04:33.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.173+0000 7fc7cbe00640 1 --2- 172.21.15.103:0/3230783337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c4106f10 0x7fc7c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:33.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.173+0000 7fc7cbe00640 1 --2- 172.21.15.103:0/3230783337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c4102790 0x7fc7c41067e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:33.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.173+0000 7fc7cbe00640 1 --2- 172.21.15.103:0/3230783337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7c4101de0 0x7fc7c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:33.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.173+0000 7fc7cbe00640 1 -- 172.21.15.103:0/3230783337 >> 172.21.15.103:0/3230783337 conn(0x7fc7c40fd8c0 msgr2=0x7fc7c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:33.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.173+0000 7fc7cbe00640 1 -- 172.21.15.103:0/3230783337 shutdown_connections 2024-09-16T02:04:33.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.173+0000 7fc7cbe00640 1 -- 172.21.15.103:0/3230783337 wait complete. 2024-09-16T02:04:33.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.174+0000 7fc7cbe00640 1 Processor -- start 2024-09-16T02:04:33.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.174+0000 7fc7cbe00640 1 -- start start 2024-09-16T02:04:33.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.174+0000 7fc7cbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c4101de0 0x7fc7c419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:33.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.174+0000 7fc7cbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c4102790 0x7fc7c419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:33.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.174+0000 7fc7cbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7c4106f10 0x7fc7c419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:33.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.174+0000 7fc7cbe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c4111f70 con 0x7fc7c4106f10 2024-09-16T02:04:33.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.174+0000 7fc7cbe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c4111df0 con 0x7fc7c4102790 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.174+0000 7fc7cbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c41120f0 con 0x7fc7c4101de0 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c4102790 0x7fc7c419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7c4106f10 0x7fc7c419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c9400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c4101de0 0x7fc7c419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c4102790 0x7fc7c419ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:36212/0 (socket says 172.21.15.103:36212) 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c8a00640 1 -- 172.21.15.103:0/2066200937 learned_addr learned my addr 172.21.15.103:0/2066200937 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c9e00640 1 -- 172.21.15.103:0/2066200937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c4101de0 msgr2=0x7fc7c419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c9e00640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c4101de0 0x7fc7c419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c9e00640 1 -- 172.21.15.103:0/2066200937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c4102790 msgr2=0x7fc7c419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c9e00640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c4102790 0x7fc7c419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c9e00640 1 -- 172.21.15.103:0/2066200937 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c419f6a0 con 0x7fc7c4106f10 2024-09-16T02:04:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c9e00640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7c4106f10 0x7fc7c419efc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009fd0 tx=0x7fc7bc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.175+0000 7fc7c2000640 1 -- 172.21.15.103:0/2066200937 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc00ce20 con 0x7fc7c4106f10 2024-09-16T02:04:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.176+0000 7fc7cbe00640 1 -- 172.21.15.103:0/2066200937 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c419f990 con 0x7fc7c4106f10 2024-09-16T02:04:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.176+0000 7fc7c2000640 1 -- 172.21.15.103:0/2066200937 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7bc005020 con 0x7fc7c4106f10 2024-09-16T02:04:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.176+0000 7fc7c2000640 1 -- 172.21.15.103:0/2066200937 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc010720 con 0x7fc7c4106f10 2024-09-16T02:04:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.176+0000 7fc7cbe00640 1 -- 172.21.15.103:0/2066200937 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c41a7270 con 0x7fc7c4106f10 2024-09-16T02:04:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.177+0000 7fc7cbe00640 1 -- 172.21.15.103:0/2066200937 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c41021c0 con 0x7fc7c4106f10 2024-09-16T02:04:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.179+0000 7fc7c2000640 1 -- 172.21.15.103:0/2066200937 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc7bc005540 con 0x7fc7c4106f10 2024-09-16T02:04:33.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.180+0000 7fc7c2000640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc7ac0a3350 0x7fc7ac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:33.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.180+0000 7fc7c2000640 1 -- 172.21.15.103:0/2066200937 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fc7bc0d0170 con 0x7fc7c4106f10 2024-09-16T02:04:33.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.180+0000 7fc7c9400640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc7ac0a3350 0x7fc7ac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:33.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.181+0000 7fc7c9400640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc7ac0a3350 0x7fc7ac0a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009bd0 tx=0x7fc7b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:33.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.183+0000 7fc7c2000640 1 -- 172.21.15.103:0/2066200937 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7bc09cd50 con 0x7fc7c4106f10 2024-09-16T02:04:33.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.363+0000 7fc7cbe00640 1 -- 172.21.15.103:0/2066200937 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc7c419ba50 con 0x7fc7c4106f10 2024-09-16T02:04:33.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.364+0000 7fc7c2000640 1 -- 172.21.15.103:0/2066200937 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc7c419ba50 con 0x7fc7c4106f10 2024-09-16T02:04:33.366 INFO:teuthology.orchestra.run.smithi103.stdout:197568495661 2024-09-16T02:04:33.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 -- 172.21.15.103:0/2066200937 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc7ac0a3350 msgr2=0x7fc7ac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:33.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc7ac0a3350 0x7fc7ac0a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009bd0 tx=0x7fc7b8005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 -- 172.21.15.103:0/2066200937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7c4106f10 msgr2=0x7fc7c419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7c4106f10 0x7fc7c419efc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009fd0 tx=0x7fc7bc00ef10 comp rx=0 tx=0).stop 2024-09-16T02:04:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 -- 172.21.15.103:0/2066200937 shutdown_connections 2024-09-16T02:04:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc7ac0a3350 0x7fc7ac0a5810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7c4106f10 0x7fc7c419efc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c4102790 0x7fc7c419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 --2- 172.21.15.103:0/2066200937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c4101de0 0x7fc7c419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 -- 172.21.15.103:0/2066200937 >> 172.21.15.103:0/2066200937 conn(0x7fc7c40fd8c0 msgr2=0x7fc7c40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 -- 172.21.15.103:0/2066200937 shutdown_connections 2024-09-16T02:04:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.366+0000 7fc7cbe00640 1 -- 172.21.15.103:0/2066200937 wait complete. 2024-09-16T02:04:33.562 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888116 got 146028888118 for osd.5 2024-09-16T02:04:33.562 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:33.751 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:33 smithi103 ceph-mon[30398]: pgmap v311: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T02:04:33.751 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:33 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2066200937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T02:04:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:33 smithi136 ceph-mon[35323]: pgmap v311: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T02:04:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:33 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2066200937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T02:04:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:33 smithi172 ceph-mon[35340]: pgmap v311: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T02:04:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:33 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2066200937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T02:04:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.083+0000 7f65de600640 1 -- 172.21.15.103:0/461459717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65d8103ff0 msgr2=0x7f65d81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.083+0000 7f65de600640 1 --2- 172.21.15.103:0/461459717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65d8103ff0 0x7f65d81043d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009900 tx=0x7f65c802a3f0 comp rx=0 tx=0).stop 2024-09-16T02:04:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.083+0000 7f65de600640 1 -- 172.21.15.103:0/461459717 shutdown_connections 2024-09-16T02:04:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.083+0000 7f65de600640 1 --2- 172.21.15.103:0/461459717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65d8109120 0x7f65d8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.083+0000 7f65de600640 1 --2- 172.21.15.103:0/461459717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d81049a0 0x7f65d81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.083+0000 7f65de600640 1 --2- 172.21.15.103:0/461459717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65d8103ff0 0x7f65d81043d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.083+0000 7f65de600640 1 -- 172.21.15.103:0/461459717 >> 172.21.15.103:0/461459717 conn(0x7f65d80ffa90 msgr2=0x7f65d8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.084+0000 7f65de600640 1 -- 172.21.15.103:0/461459717 shutdown_connections 2024-09-16T02:04:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.084+0000 7f65de600640 1 -- 172.21.15.103:0/461459717 wait complete. 2024-09-16T02:04:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.084+0000 7f65de600640 1 Processor -- start 2024-09-16T02:04:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65de600640 1 -- start start 2024-09-16T02:04:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65de600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65d8103ff0 0x7f65d819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65de600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65d81049a0 0x7f65d819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65de600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8109120 0x7f65d819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65de600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8114190 con 0x7f65d8103ff0 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65de600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8114010 con 0x7f65d81049a0 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65de600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8114310 con 0x7f65d8109120 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65d7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65d8103ff0 0x7f65d819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65d7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65d8103ff0 0x7f65d819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41764/0 (socket says 172.21.15.103:41764) 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65d7400640 1 -- 172.21.15.103:0/1594832623 learned_addr learned my addr 172.21.15.103:0/1594832623 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65d7e00640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8109120 0x7f65d819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.085+0000 7f65d6a00640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65d81049a0 0x7f65d819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.086+0000 7f65d7400640 1 -- 172.21.15.103:0/1594832623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8109120 msgr2=0x7f65d819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.086+0000 7f65d7400640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8109120 0x7f65d819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.086+0000 7f65d7400640 1 -- 172.21.15.103:0/1594832623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65d81049a0 msgr2=0x7f65d819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.086+0000 7f65d7400640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65d81049a0 0x7f65d819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.086+0000 7f65d7400640 1 -- 172.21.15.103:0/1594832623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d819f640 con 0x7f65d8103ff0 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.086+0000 7f65d7e00640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8109120 0x7f65d819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.086+0000 7f65d7400640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65d8103ff0 0x7f65d819a690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65c8004770 tx=0x7f65c8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.087+0000 7f65cfe00640 1 -- 172.21.15.103:0/1594832623 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c803f070 con 0x7f65d8103ff0 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.087+0000 7f65cfe00640 1 -- 172.21.15.103:0/1594832623 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c8031ca0 con 0x7f65d8103ff0 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.087+0000 7f65de600640 1 -- 172.21.15.103:0/1594832623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d819f8d0 con 0x7f65d8103ff0 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.087+0000 7f65cfe00640 1 -- 172.21.15.103:0/1594832623 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8035030 con 0x7f65d8103ff0 2024-09-16T02:04:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.087+0000 7f65de600640 1 -- 172.21.15.103:0/1594832623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81a7210 con 0x7f65d8103ff0 2024-09-16T02:04:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.088+0000 7f65de600640 1 -- 172.21.15.103:0/1594832623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d81043d0 con 0x7f65d8103ff0 2024-09-16T02:04:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.092+0000 7f65cfe00640 1 -- 172.21.15.103:0/1594832623 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f65c8042070 con 0x7f65d8103ff0 2024-09-16T02:04:34.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.093+0000 7f65cfe00640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65bc0a3350 0x7f65bc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:34.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.093+0000 7f65cfe00640 1 -- 172.21.15.103:0/1594832623 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f65c80f0150 con 0x7f65d8103ff0 2024-09-16T02:04:34.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.094+0000 7f65d6a00640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65bc0a3350 0x7f65bc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:34.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.095+0000 7f65d6a00640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65bc0a3350 0x7f65bc0a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65d80fffb0 tx=0x7f65c400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:34.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.098+0000 7f65cfe00640 1 -- 172.21.15.103:0/1594832623 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65c80bc6b0 con 0x7f65d8103ff0 2024-09-16T02:04:34.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.289+0000 7f65de600640 1 -- 172.21.15.103:0/1594832623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f65d810c300 con 0x7f65d8103ff0 2024-09-16T02:04:34.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.290+0000 7f65cfe00640 1 -- 172.21.15.103:0/1594832623 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f65c80c09b0 con 0x7f65d8103ff0 2024-09-16T02:04:34.292 INFO:teuthology.orchestra.run.smithi103.stdout:68719476807 2024-09-16T02:04:34.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 -- 172.21.15.103:0/1594832623 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65bc0a3350 msgr2=0x7f65bc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65bc0a3350 0x7f65bc0a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65d80fffb0 tx=0x7f65c400b390 comp rx=0 tx=0).stop 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 -- 172.21.15.103:0/1594832623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65d8103ff0 msgr2=0x7f65d819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65d8103ff0 0x7f65d819a690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65c8004770 tx=0x7f65c8003c10 comp rx=0 tx=0).stop 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 -- 172.21.15.103:0/1594832623 shutdown_connections 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f65bc0a3350 0x7f65bc0a5810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8109120 0x7f65d819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65d81049a0 0x7f65d819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 --2- 172.21.15.103:0/1594832623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65d8103ff0 0x7f65d819a690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 -- 172.21.15.103:0/1594832623 >> 172.21.15.103:0/1594832623 conn(0x7f65d80ffa90 msgr2=0x7f65d8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.293+0000 7f65de600640 1 -- 172.21.15.103:0/1594832623 shutdown_connections 2024-09-16T02:04:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.294+0000 7f65de600640 1 -- 172.21.15.103:0/1594832623 wait complete. 2024-09-16T02:04:34.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:34 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3827506310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T02:04:34.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:34 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1594832623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T02:04:35.034 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:34 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3827506310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T02:04:35.034 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:34 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1594832623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T02:04:35.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:34 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3827506310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T02:04:35.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:34 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1594832623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T02:04:35.193 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037792 got 257698037793 for osd.9 2024-09-16T02:04:35.193 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:35.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.955+0000 7f8021800640 1 -- 172.21.15.103:0/2274516195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801c101de0 msgr2=0x7f801c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:35.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.955+0000 7f8021800640 1 --2- 172.21.15.103:0/2274516195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801c101de0 0x7f801c1021c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f800c009ca0 tx=0x7f800c02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.955+0000 7f8021800640 1 -- 172.21.15.103:0/2274516195 shutdown_connections 2024-09-16T02:04:35.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.955+0000 7f8021800640 1 --2- 172.21.15.103:0/2274516195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c106f10 0x7f801c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.955+0000 7f8021800640 1 --2- 172.21.15.103:0/2274516195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f801c102790 0x7f801c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.955+0000 7f8021800640 1 --2- 172.21.15.103:0/2274516195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801c101de0 0x7f801c1021c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.955+0000 7f8021800640 1 -- 172.21.15.103:0/2274516195 >> 172.21.15.103:0/2274516195 conn(0x7f801c0fd8c0 msgr2=0x7f801c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:35.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.955+0000 7f8021800640 1 -- 172.21.15.103:0/2274516195 shutdown_connections 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.955+0000 7f8021800640 1 -- 172.21.15.103:0/2274516195 wait complete. 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.956+0000 7f8021800640 1 Processor -- start 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.956+0000 7f8021800640 1 -- start start 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.956+0000 7f8021800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f801c101de0 0x7f801c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.956+0000 7f8021800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801c102790 0x7f801c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.956+0000 7f8021800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c106f10 0x7f801c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.956+0000 7f8021800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f801c111fc0 con 0x7f801c102790 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.956+0000 7f8021800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f801c111e40 con 0x7f801c106f10 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.956+0000 7f8021800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f801c112140 con 0x7f801c101de0 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.956+0000 7f801a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801c102790 0x7f801c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:35.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f801a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801c102790 0x7f801c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41732/0 (socket says 172.21.15.103:41732) 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f801a000640 1 -- 172.21.15.103:0/3827506310 learned_addr learned my addr 172.21.15.103:0/3827506310 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f801aa00640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f801c101de0 0x7f801c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f801b400640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c106f10 0x7f801c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f801a000640 1 -- 172.21.15.103:0/3827506310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f801c101de0 msgr2=0x7f801c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f801a000640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f801c101de0 0x7f801c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f801a000640 1 -- 172.21.15.103:0/3827506310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c106f10 msgr2=0x7f801c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f801a000640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c106f10 0x7f801c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f801a000640 1 -- 172.21.15.103:0/3827506310 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c19f6a0 con 0x7f801c102790 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f801a000640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801c102790 0x7f801c19ac30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f800800e9f0 tx=0x7f800800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f8013400640 1 -- 172.21.15.103:0/3827506310 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800800cd80 con 0x7f801c102790 2024-09-16T02:04:35.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.957+0000 7f8013400640 1 -- 172.21.15.103:0/3827506310 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8008005500 con 0x7f801c102790 2024-09-16T02:04:35.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.958+0000 7f8021800640 1 -- 172.21.15.103:0/3827506310 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c19f990 con 0x7f801c102790 2024-09-16T02:04:35.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.958+0000 7f8013400640 1 -- 172.21.15.103:0/3827506310 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008010690 con 0x7f801c102790 2024-09-16T02:04:35.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.958+0000 7f8021800640 1 -- 172.21.15.103:0/3827506310 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c1a7270 con 0x7f801c102790 2024-09-16T02:04:35.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.959+0000 7f8021800640 1 -- 172.21.15.103:0/3827506310 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c1021c0 con 0x7f801c102790 2024-09-16T02:04:35.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.961+0000 7f8013400640 1 -- 172.21.15.103:0/3827506310 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f80080046c0 con 0x7f801c102790 2024-09-16T02:04:35.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.962+0000 7f8013400640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f80000a32d0 0x7f80000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:35.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.963+0000 7f8013400640 1 -- 172.21.15.103:0/3827506310 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f80080cfef0 con 0x7f801c102790 2024-09-16T02:04:35.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.963+0000 7f801aa00640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f80000a32d0 0x7f80000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:35.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.963+0000 7f801aa00640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f80000a32d0 0x7f80000a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f800c006010 tx=0x7f800c009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:35.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:33.966+0000 7f8013400640 1 -- 172.21.15.103:0/3827506310 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f800809ca40 con 0x7f801c102790 2024-09-16T02:04:35.280 INFO:teuthology.orchestra.run.smithi103.stdout:34359738449 2024-09-16T02:04:35.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.161+0000 7f8021800640 1 -- 172.21.15.103:0/3827506310 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f801c1a74d0 con 0x7f801c102790 2024-09-16T02:04:35.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.162+0000 7f8013400640 1 -- 172.21.15.103:0/3827506310 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f800809c340 con 0x7f801c102790 2024-09-16T02:04:35.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.164+0000 7f8021800640 1 -- 172.21.15.103:0/3827506310 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f80000a32d0 msgr2=0x7f80000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:35.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.164+0000 7f8021800640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f80000a32d0 0x7f80000a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f800c006010 tx=0x7f800c009820 comp rx=0 tx=0).stop 2024-09-16T02:04:35.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.164+0000 7f8021800640 1 -- 172.21.15.103:0/3827506310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801c102790 msgr2=0x7f801c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:35.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.164+0000 7f8021800640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801c102790 0x7f801c19ac30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f800800e9f0 tx=0x7f800800edf0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.164+0000 7f8021800640 1 -- 172.21.15.103:0/3827506310 shutdown_connections 2024-09-16T02:04:35.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.164+0000 7f8021800640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f80000a32d0 0x7f80000a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.164+0000 7f8021800640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c106f10 0x7f801c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.164+0000 7f8021800640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801c102790 0x7f801c19ac30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.164+0000 7f8021800640 1 --2- 172.21.15.103:0/3827506310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f801c101de0 0x7f801c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:35.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.164+0000 7f8021800640 1 -- 172.21.15.103:0/3827506310 >> 172.21.15.103:0/3827506310 conn(0x7f801c0fd8c0 msgr2=0x7f801c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:35.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.165+0000 7f8021800640 1 -- 172.21.15.103:0/3827506310 shutdown_connections 2024-09-16T02:04:35.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:34.165+0000 7f8021800640 1 -- 172.21.15.103:0/3827506310 wait complete. 2024-09-16T02:04:35.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:35 smithi103 ceph-mon[30398]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 129 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T02:04:36.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:35 smithi136 ceph-mon[35323]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 129 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T02:04:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:35 smithi172 ceph-mon[35340]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 129 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T02:04:36.073 INFO:tasks.cephadm.ceph_manager.ceph:need seq 51539607626 got 51539607629 for osd.1 2024-09-16T02:04:36.073 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:36.588 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495658 got 197568495661 for osd.7 2024-09-16T02:04:36.589 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:36.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.795+0000 7f6cdac00640 1 -- 172.21.15.103:0/812706130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd4102790 msgr2=0x7f6cd41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:36.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.795+0000 7f6cdac00640 1 --2- 172.21.15.103:0/812706130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd4102790 0x7f6cd41067e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009900 tx=0x7f6cbc02a410 comp rx=0 tx=0).stop 2024-09-16T02:04:36.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.795+0000 7f6cdac00640 1 -- 172.21.15.103:0/812706130 shutdown_connections 2024-09-16T02:04:36.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.795+0000 7f6cdac00640 1 --2- 172.21.15.103:0/812706130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cd4106f10 0x7f6cd410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:36.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.795+0000 7f6cdac00640 1 --2- 172.21.15.103:0/812706130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd4102790 0x7f6cd41067e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:36.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.795+0000 7f6cdac00640 1 --2- 172.21.15.103:0/812706130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cd4101de0 0x7f6cd41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:36.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.795+0000 7f6cdac00640 1 -- 172.21.15.103:0/812706130 >> 172.21.15.103:0/812706130 conn(0x7f6cd40fd8c0 msgr2=0x7f6cd40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:36.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.795+0000 7f6cdac00640 1 -- 172.21.15.103:0/812706130 shutdown_connections 2024-09-16T02:04:36.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.795+0000 7f6cdac00640 1 -- 172.21.15.103:0/812706130 wait complete. 2024-09-16T02:04:36.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.796+0000 7f6cdac00640 1 Processor -- start 2024-09-16T02:04:36.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6cdac00640 1 -- start start 2024-09-16T02:04:36.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6cdac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cd4101de0 0x7f6cd419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:36.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6cdac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd4102790 0x7f6cd419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:36.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6cdac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cd4106f10 0x7f6cd419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:36.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6cdac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4111f30 con 0x7f6cd4102790 2024-09-16T02:04:36.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6cdac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4111db0 con 0x7f6cd4101de0 2024-09-16T02:04:36.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6cdac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41120b0 con 0x7f6cd4106f10 2024-09-16T02:04:36.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6ccb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd4102790 0x7f6cd419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:36.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6ccb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd4102790 0x7f6cd419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41786/0 (socket says 172.21.15.103:41786) 2024-09-16T02:04:36.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6ccb400640 1 -- 172.21.15.103:0/1353030767 learned_addr learned my addr 172.21.15.103:0/1353030767 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:36.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.797+0000 7f6cd3e00640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cd4101de0 0x7f6cd419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:36.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.798+0000 7f6cd8c00640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cd4106f10 0x7f6cd419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.798+0000 7f6ccb400640 1 -- 172.21.15.103:0/1353030767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cd4106f10 msgr2=0x7f6cd419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.798+0000 7f6ccb400640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cd4106f10 0x7f6cd419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.798+0000 7f6ccb400640 1 -- 172.21.15.103:0/1353030767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cd4101de0 msgr2=0x7f6cd419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.798+0000 7f6ccb400640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cd4101de0 0x7f6cd419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.798+0000 7f6ccb400640 1 -- 172.21.15.103:0/1353030767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd419f600 con 0x7f6cd4102790 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.798+0000 7f6ccb400640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd4102790 0x7f6cd419ab90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc006010 tx=0x7f6cbc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.798+0000 7f6cd1600640 1 -- 172.21.15.103:0/1353030767 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc03f030 con 0x7f6cd4102790 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.799+0000 7f6cdac00640 1 -- 172.21.15.103:0/1353030767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd419f890 con 0x7f6cd4102790 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.799+0000 7f6cd1600640 1 -- 172.21.15.103:0/1353030767 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cbc003ce0 con 0x7f6cd4102790 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.799+0000 7f6cd1600640 1 -- 172.21.15.103:0/1353030767 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc035070 con 0x7f6cd4102790 2024-09-16T02:04:36.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.799+0000 7f6cdac00640 1 -- 172.21.15.103:0/1353030767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd41a71d0 con 0x7f6cd4102790 2024-09-16T02:04:36.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.800+0000 7f6cdac00640 1 -- 172.21.15.103:0/1353030767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd41021c0 con 0x7f6cd4102790 2024-09-16T02:04:36.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.803+0000 7f6cd1600640 1 -- 172.21.15.103:0/1353030767 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6cbc042030 con 0x7f6cd4102790 2024-09-16T02:04:36.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.804+0000 7f6cd1600640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6cb80a32d0 0x7f6cb80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:36.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.805+0000 7f6cd1600640 1 -- 172.21.15.103:0/1353030767 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f6cbc0f0210 con 0x7f6cd4102790 2024-09-16T02:04:36.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.805+0000 7f6cd3e00640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6cb80a32d0 0x7f6cb80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:36.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.805+0000 7f6cd3e00640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6cb80a32d0 0x7f6cb80a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400bbe0 tx=0x7f6cc400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:36.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.807+0000 7f6cd1600640 1 -- 172.21.15.103:0/1353030767 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6cbc0bc7d0 con 0x7f6cd4102790 2024-09-16T02:04:36.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.990+0000 7f6cdac00640 1 -- 172.21.15.103:0/1353030767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6cd41a73a0 con 0x7f6cd4102790 2024-09-16T02:04:36.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.990+0000 7f6cd1600640 1 -- 172.21.15.103:0/1353030767 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6cbc0c0ad0 con 0x7f6cd4102790 2024-09-16T02:04:36.992 INFO:teuthology.orchestra.run.smithi103.stdout:176093659187 2024-09-16T02:04:36.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.992+0000 7f6cdac00640 1 -- 172.21.15.103:0/1353030767 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6cb80a32d0 msgr2=0x7f6cb80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.992+0000 7f6cdac00640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6cb80a32d0 0x7f6cb80a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400bbe0 tx=0x7f6cc400b390 comp rx=0 tx=0).stop 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.992+0000 7f6cdac00640 1 -- 172.21.15.103:0/1353030767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd4102790 msgr2=0x7f6cd419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.992+0000 7f6cdac00640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd4102790 0x7f6cd419ab90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc006010 tx=0x7f6cbc0042f0 comp rx=0 tx=0).stop 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.992+0000 7f6cdac00640 1 -- 172.21.15.103:0/1353030767 shutdown_connections 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.992+0000 7f6cdac00640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6cb80a32d0 0x7f6cb80a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.992+0000 7f6cdac00640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cd4106f10 0x7f6cd419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.992+0000 7f6cdac00640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd4102790 0x7f6cd419ab90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.992+0000 7f6cdac00640 1 --2- 172.21.15.103:0/1353030767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cd4101de0 0x7f6cd419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.992+0000 7f6cdac00640 1 -- 172.21.15.103:0/1353030767 >> 172.21.15.103:0/1353030767 conn(0x7f6cd40fd8c0 msgr2=0x7f6cd40fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.993+0000 7f6cdac00640 1 -- 172.21.15.103:0/1353030767 shutdown_connections 2024-09-16T02:04:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:36.993+0000 7f6cdac00640 1 -- 172.21.15.103:0/1353030767 wait complete. 2024-09-16T02:04:37.264 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476805 got 68719476807 for osd.2 2024-09-16T02:04:37.265 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:37.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:37 smithi103 ceph-mon[30398]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3 op/s; 143 B/s, 1 keys/s, 2 objects/s recovering 2024-09-16T02:04:37.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:37 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1353030767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T02:04:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.822+0000 7fab1e600640 1 -- 172.21.15.103:0/2861969390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab18101de0 msgr2=0x7fab181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.822+0000 7fab1e600640 1 --2- 172.21.15.103:0/2861969390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab18101de0 0x7fab181021c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fab08009900 tx=0x7fab0802a410 comp rx=0 tx=0).stop 2024-09-16T02:04:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.822+0000 7fab1e600640 1 -- 172.21.15.103:0/2861969390 shutdown_connections 2024-09-16T02:04:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.822+0000 7fab1e600640 1 --2- 172.21.15.103:0/2861969390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab18106f10 0x7fab1810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.822+0000 7fab1e600640 1 --2- 172.21.15.103:0/2861969390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab18102790 0x7fab181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.822+0000 7fab1e600640 1 --2- 172.21.15.103:0/2861969390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab18101de0 0x7fab181021c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.822+0000 7fab1e600640 1 -- 172.21.15.103:0/2861969390 >> 172.21.15.103:0/2861969390 conn(0x7fab180fd8c0 msgr2=0x7fab180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.823+0000 7fab1e600640 1 -- 172.21.15.103:0/2861969390 shutdown_connections 2024-09-16T02:04:37.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.823+0000 7fab1e600640 1 -- 172.21.15.103:0/2861969390 wait complete. 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.823+0000 7fab1e600640 1 Processor -- start 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab1e600640 1 -- start start 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab1e600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab18101de0 0x7fab1819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab1e600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab18102790 0x7fab1819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab1e600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab18106f10 0x7fab1819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab1e600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab18111fc0 con 0x7fab18106f10 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab1e600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab18111e40 con 0x7fab18101de0 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab1e600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fab18112140 con 0x7fab18102790 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab17e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab18106f10 0x7fab1819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab17400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab18101de0 0x7fab1819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab17e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab18106f10 0x7fab1819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41808/0 (socket says 172.21.15.103:41808) 2024-09-16T02:04:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.824+0000 7fab17e00640 1 -- 172.21.15.103:0/3814911813 learned_addr learned my addr 172.21.15.103:0/3814911813 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab0ea00640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab18102790 0x7fab1819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab17e00640 1 -- 172.21.15.103:0/3814911813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab18102790 msgr2=0x7fab1819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab17e00640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab18102790 0x7fab1819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab17e00640 1 -- 172.21.15.103:0/3814911813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab18101de0 msgr2=0x7fab1819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab17e00640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab18101de0 0x7fab1819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab17e00640 1 -- 172.21.15.103:0/3814911813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1819f5e0 con 0x7fab18106f10 2024-09-16T02:04:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab17400640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab18101de0 0x7fab1819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:37.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab17e00640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab18106f10 0x7fab1819ef00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab1000eb10 tx=0x7fab1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:37.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab14c00640 1 -- 172.21.15.103:0/3814911813 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10004b20 con 0x7fab18106f10 2024-09-16T02:04:37.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab14c00640 1 -- 172.21.15.103:0/3814911813 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab10004cc0 con 0x7fab18106f10 2024-09-16T02:04:37.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab14c00640 1 -- 172.21.15.103:0/3814911813 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10010760 con 0x7fab18106f10 2024-09-16T02:04:37.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.825+0000 7fab1e600640 1 -- 172.21.15.103:0/3814911813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1819f8d0 con 0x7fab18106f10 2024-09-16T02:04:37.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.826+0000 7fab1e600640 1 -- 172.21.15.103:0/3814911813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab181a71b0 con 0x7fab18106f10 2024-09-16T02:04:37.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.827+0000 7fab1e600640 1 -- 172.21.15.103:0/3814911813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab181021c0 con 0x7fab18106f10 2024-09-16T02:04:37.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.829+0000 7fab14c00640 1 -- 172.21.15.103:0/3814911813 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fab10005000 con 0x7fab18106f10 2024-09-16T02:04:37.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.830+0000 7fab14c00640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faafc0a3350 0x7faafc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:37.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.830+0000 7fab14c00640 1 -- 172.21.15.103:0/3814911813 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fab100cffa0 con 0x7fab18106f10 2024-09-16T02:04:37.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.830+0000 7fab17400640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faafc0a3350 0x7faafc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:37.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.831+0000 7fab17400640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faafc0a3350 0x7faafc0a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab08000c00 tx=0x7fab08031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:37.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:37.834+0000 7fab14c00640 1 -- 172.21.15.103:0/3814911813 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab1009cb50 con 0x7fab18106f10 2024-09-16T02:04:38.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.017+0000 7fab1e600640 1 -- 172.21.15.103:0/3814911813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fab181a7490 con 0x7fab18106f10 2024-09-16T02:04:38.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.018+0000 7fab14c00640 1 -- 172.21.15.103:0/3814911813 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fab1009c450 con 0x7fab18106f10 2024-09-16T02:04:38.020 INFO:teuthology.orchestra.run.smithi103.stdout:274877906973 2024-09-16T02:04:38.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 -- 172.21.15.103:0/3814911813 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faafc0a3350 msgr2=0x7faafc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:38.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faafc0a3350 0x7faafc0a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab08000c00 tx=0x7fab08031510 comp rx=0 tx=0).stop 2024-09-16T02:04:38.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 -- 172.21.15.103:0/3814911813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab18106f10 msgr2=0x7fab1819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:38.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab18106f10 0x7fab1819ef00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab1000eb10 tx=0x7fab1000c2d0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 -- 172.21.15.103:0/3814911813 shutdown_connections 2024-09-16T02:04:38.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faafc0a3350 0x7faafc0a5810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab18106f10 0x7fab1819ef00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab18102790 0x7fab1819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 --2- 172.21.15.103:0/3814911813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab18101de0 0x7fab1819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 -- 172.21.15.103:0/3814911813 >> 172.21.15.103:0/3814911813 conn(0x7fab180fd8c0 msgr2=0x7fab180febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:38.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.020+0000 7fab1e600640 1 -- 172.21.15.103:0/3814911813 shutdown_connections 2024-09-16T02:04:38.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.021+0000 7fab1e600640 1 -- 172.21.15.103:0/3814911813 wait complete. 2024-09-16T02:04:38.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:37 smithi136 ceph-mon[35323]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3 op/s; 143 B/s, 1 keys/s, 2 objects/s recovering 2024-09-16T02:04:38.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:37 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1353030767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T02:04:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:37 smithi172 ceph-mon[35340]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3 op/s; 143 B/s, 1 keys/s, 2 objects/s recovering 2024-09-16T02:04:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:37 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1353030767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T02:04:38.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.584+0000 7f902fe00640 1 -- 172.21.15.103:0/2531804039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9030108f60 msgr2=0x7f9030110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:38.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.584+0000 7f902fe00640 1 --2- 172.21.15.103:0/2531804039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9030108f60 0x7f9030110ae0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9028009ca0 tx=0x7f902802a8c0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.584+0000 7f902fe00640 1 -- 172.21.15.103:0/2531804039 shutdown_connections 2024-09-16T02:04:38.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.584+0000 7f902fe00640 1 --2- 172.21.15.103:0/2531804039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9030108f60 0x7f9030110ae0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.584+0000 7f902fe00640 1 --2- 172.21.15.103:0/2531804039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90301047e0 0x7f9030108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.584+0000 7f902fe00640 1 --2- 172.21.15.103:0/2531804039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030103e30 0x7f9030104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.584+0000 7f902fe00640 1 -- 172.21.15.103:0/2531804039 >> 172.21.15.103:0/2531804039 conn(0x7f90300ff910 msgr2=0x7f9030101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:38.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.584+0000 7f902fe00640 1 -- 172.21.15.103:0/2531804039 shutdown_connections 2024-09-16T02:04:38.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.584+0000 7f902fe00640 1 -- 172.21.15.103:0/2531804039 wait complete. 2024-09-16T02:04:38.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902fe00640 1 Processor -- start 2024-09-16T02:04:38.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902fe00640 1 -- start start 2024-09-16T02:04:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9030103e30 0x7f903019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90301047e0 0x7f903019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030108f60 0x7f903019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9030113fc0 con 0x7f9030103e30 2024-09-16T02:04:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9030113e40 con 0x7f90301047e0 2024-09-16T02:04:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9030114140 con 0x7f9030108f60 2024-09-16T02:04:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9030103e30 0x7f903019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030108f60 0x7f903019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9030103e30 0x7f903019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33552/0 (socket says 172.21.15.103:33552) 2024-09-16T02:04:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030108f60 0x7f903019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:51262/0 (socket says 172.21.15.103:51262) 2024-09-16T02:04:38.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.586+0000 7f902ea00640 1 -- 172.21.15.103:0/2291855222 learned_addr learned my addr 172.21.15.103:0/2291855222 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:38.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.587+0000 7f9027e00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90301047e0 0x7f903019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:38.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.587+0000 7f902f400640 1 -- 172.21.15.103:0/2291855222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90301047e0 msgr2=0x7f903019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:38.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.587+0000 7f902f400640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90301047e0 0x7f903019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.587+0000 7f902f400640 1 -- 172.21.15.103:0/2291855222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9030103e30 msgr2=0x7f903019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:38.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.587+0000 7f902f400640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9030103e30 0x7f903019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.587+0000 7f902f400640 1 -- 172.21.15.103:0/2291855222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903019f3d0 con 0x7f9030108f60 2024-09-16T02:04:38.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.587+0000 7f902ea00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9030103e30 0x7f903019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:38.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.587+0000 7f902f400640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030108f60 0x7f903019ecf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f902802ade0 tx=0x7f90280319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:38.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.587+0000 7f9034a00640 1 -- 172.21.15.103:0/2291855222 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028003b70 con 0x7f9030108f60 2024-09-16T02:04:38.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.587+0000 7f9034a00640 1 -- 172.21.15.103:0/2291855222 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9028003d10 con 0x7f9030108f60 2024-09-16T02:04:38.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.588+0000 7f902fe00640 1 -- 172.21.15.103:0/2291855222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903019f660 con 0x7f9030108f60 2024-09-16T02:04:38.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.588+0000 7f9034a00640 1 -- 172.21.15.103:0/2291855222 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902802fc40 con 0x7f9030108f60 2024-09-16T02:04:38.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.588+0000 7f902fe00640 1 -- 172.21.15.103:0/2291855222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90301a6fa0 con 0x7f9030108f60 2024-09-16T02:04:38.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.589+0000 7f902fe00640 1 -- 172.21.15.103:0/2291855222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9030105310 con 0x7f9030108f60 2024-09-16T02:04:38.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.590+0000 7f9034a00640 1 -- 172.21.15.103:0/2291855222 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9028004050 con 0x7f9030108f60 2024-09-16T02:04:38.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.592+0000 7f9034a00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90140a32d0 0x7f90140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:38.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.592+0000 7f9034a00640 1 -- 172.21.15.103:0/2291855222 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f90280f0950 con 0x7f9030108f60 2024-09-16T02:04:38.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.592+0000 7f902ea00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90140a32d0 0x7f90140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:38.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.593+0000 7f902ea00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90140a32d0 0x7f90140a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9020011550 tx=0x7f9020005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:38.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.596+0000 7f9034a00640 1 -- 172.21.15.103:0/2291855222 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90280bd5b0 con 0x7f9030108f60 2024-09-16T02:04:38.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:38 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3814911813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T02:04:38.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.783+0000 7f902fe00640 1 -- 172.21.15.103:0/2291855222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9030104210 con 0x7f9030108f60 2024-09-16T02:04:38.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.784+0000 7f9034a00640 1 -- 172.21.15.103:0/2291855222 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9030104210 con 0x7f9030108f60 2024-09-16T02:04:38.786 INFO:teuthology.orchestra.run.smithi103.stdout:124554051646 2024-09-16T02:04:38.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 -- 172.21.15.103:0/2291855222 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90140a32d0 msgr2=0x7f90140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:38.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90140a32d0 0x7f90140a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9020011550 tx=0x7f9020005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:38.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 -- 172.21.15.103:0/2291855222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030108f60 msgr2=0x7f903019ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:38.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030108f60 0x7f903019ecf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f902802ade0 tx=0x7f90280319f0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 -- 172.21.15.103:0/2291855222 shutdown_connections 2024-09-16T02:04:38.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90140a32d0 0x7f90140a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030108f60 0x7f903019ecf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90301047e0 0x7f903019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 --2- 172.21.15.103:0/2291855222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9030103e30 0x7f903019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:38.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 -- 172.21.15.103:0/2291855222 >> 172.21.15.103:0/2291855222 conn(0x7f90300ff910 msgr2=0x7f9030100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:38.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 -- 172.21.15.103:0/2291855222 shutdown_connections 2024-09-16T02:04:38.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:38.786+0000 7f902fe00640 1 -- 172.21.15.103:0/2291855222 wait complete. 2024-09-16T02:04:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:38 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3814911813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T02:04:39.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:38 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3814911813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T02:04:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.423+0000 7fd675c00640 1 -- 172.21.15.103:0/3693203867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6701049c0 msgr2=0x7fd670108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.423+0000 7fd675c00640 1 --2- 172.21.15.103:0/3693203867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6701049c0 0x7fd670108a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009900 tx=0x7fd65c02a3f0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.423+0000 7fd675c00640 1 -- 172.21.15.103:0/3693203867 shutdown_connections 2024-09-16T02:04:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.423+0000 7fd675c00640 1 --2- 172.21.15.103:0/3693203867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd670109140 0x7fd670110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.423+0000 7fd675c00640 1 --2- 172.21.15.103:0/3693203867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6701049c0 0x7fd670108a10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.423+0000 7fd675c00640 1 --2- 172.21.15.103:0/3693203867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd670104010 0x7fd6701043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.423+0000 7fd675c00640 1 -- 172.21.15.103:0/3693203867 >> 172.21.15.103:0/3693203867 conn(0x7fd6700ffa50 msgr2=0x7fd670101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.423+0000 7fd675c00640 1 -- 172.21.15.103:0/3693203867 shutdown_connections 2024-09-16T02:04:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.423+0000 7fd675c00640 1 -- 172.21.15.103:0/3693203867 wait complete. 2024-09-16T02:04:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.424+0000 7fd675c00640 1 Processor -- start 2024-09-16T02:04:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.424+0000 7fd675c00640 1 -- start start 2024-09-16T02:04:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.424+0000 7fd675c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd670104010 0x7fd67019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.424+0000 7fd675c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6701049c0 0x7fd67019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd675c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd670109140 0x7fd67019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd675c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701141b0 con 0x7fd6701049c0 2024-09-16T02:04:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd675c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd670114030 con 0x7fd670104010 2024-09-16T02:04:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd675c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd670114330 con 0x7fd670109140 2024-09-16T02:04:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd66e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6701049c0 0x7fd67019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd66ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd670104010 0x7fd67019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd66e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6701049c0 0x7fd67019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33578/0 (socket says 172.21.15.103:33578) 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd66e000640 1 -- 172.21.15.103:0/1169990120 learned_addr learned my addr 172.21.15.103:0/1169990120 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd66f400640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd670109140 0x7fd67019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd66ea00640 1 -- 172.21.15.103:0/1169990120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd670109140 msgr2=0x7fd67019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd66ea00640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd670109140 0x7fd67019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd66ea00640 1 -- 172.21.15.103:0/1169990120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6701049c0 msgr2=0x7fd67019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd66ea00640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6701049c0 0x7fd67019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.425+0000 7fd66ea00640 1 -- 172.21.15.103:0/1169990120 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67019f6c0 con 0x7fd670104010 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.426+0000 7fd66ea00640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd670104010 0x7fd67019a710 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd66000e9f0 tx=0x7fd66000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.426+0000 7fd667400640 1 -- 172.21.15.103:0/1169990120 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66000ccc0 con 0x7fd670104010 2024-09-16T02:04:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.426+0000 7fd675c00640 1 -- 172.21.15.103:0/1169990120 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67019f9b0 con 0x7fd670104010 2024-09-16T02:04:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.426+0000 7fd667400640 1 -- 172.21.15.103:0/1169990120 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd660010910 con 0x7fd670104010 2024-09-16T02:04:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.426+0000 7fd667400640 1 -- 172.21.15.103:0/1169990120 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660018960 con 0x7fd670104010 2024-09-16T02:04:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.426+0000 7fd675c00640 1 -- 172.21.15.103:0/1169990120 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6701a7290 con 0x7fd670104010 2024-09-16T02:04:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.427+0000 7fd675c00640 1 -- 172.21.15.103:0/1169990120 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6701043f0 con 0x7fd670104010 2024-09-16T02:04:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.430+0000 7fd667400640 1 -- 172.21.15.103:0/1169990120 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd660010ac0 con 0x7fd670104010 2024-09-16T02:04:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.432+0000 7fd667400640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd6540a3350 0x7fd6540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.432+0000 7fd667400640 1 -- 172.21.15.103:0/1169990120 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fd6600cff90 con 0x7fd670104010 2024-09-16T02:04:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.432+0000 7fd66e000640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd6540a3350 0x7fd6540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.433+0000 7fd66e000640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd6540a3350 0x7fd6540a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0096c0 tx=0x7fd65c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.433+0000 7fd667400640 1 -- 172.21.15.103:0/1169990120 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd66009c490 con 0x7fd670104010 2024-09-16T02:04:39.539 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659184 got 176093659187 for osd.6 2024-09-16T02:04:39.539 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:39.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.620+0000 7fd675c00640 1 -- 172.21.15.103:0/1169990120 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd67010c380 con 0x7fd670104010 2024-09-16T02:04:39.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.621+0000 7fd667400640 1 -- 172.21.15.103:0/1169990120 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd6600a0790 con 0x7fd670104010 2024-09-16T02:04:39.623 INFO:teuthology.orchestra.run.smithi103.stdout:94489280579 2024-09-16T02:04:39.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.623+0000 7fd675c00640 1 -- 172.21.15.103:0/1169990120 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd6540a3350 msgr2=0x7fd6540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:39.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.623+0000 7fd675c00640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd6540a3350 0x7fd6540a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0096c0 tx=0x7fd65c0046c0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.623+0000 7fd675c00640 1 -- 172.21.15.103:0/1169990120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd670104010 msgr2=0x7fd67019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:39.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.623+0000 7fd675c00640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd670104010 0x7fd67019a710 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd66000e9f0 tx=0x7fd66000eeb0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.623+0000 7fd675c00640 1 -- 172.21.15.103:0/1169990120 shutdown_connections 2024-09-16T02:04:39.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.623+0000 7fd675c00640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd6540a3350 0x7fd6540a5810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.623+0000 7fd675c00640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd670109140 0x7fd67019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.623+0000 7fd675c00640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6701049c0 0x7fd67019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.623+0000 7fd675c00640 1 --2- 172.21.15.103:0/1169990120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd670104010 0x7fd67019a710 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:39.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.623+0000 7fd675c00640 1 -- 172.21.15.103:0/1169990120 >> 172.21.15.103:0/1169990120 conn(0x7fd6700ffa50 msgr2=0x7fd670100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:39.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.624+0000 7fd675c00640 1 -- 172.21.15.103:0/1169990120 shutdown_connections 2024-09-16T02:04:39.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.624+0000 7fd675c00640 1 -- 172.21.15.103:0/1169990120 wait complete. 2024-09-16T02:04:39.880 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:39 smithi103 ceph-mon[30398]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 116 B/s, 1 keys/s, 1 objects/s recovering 2024-09-16T02:04:39.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:39 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2291855222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T02:04:39.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:39 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2460423961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T02:04:39.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:39 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1169990120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T02:04:39.924 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:39 smithi136 ceph-mon[35323]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 116 B/s, 1 keys/s, 1 objects/s recovering 2024-09-16T02:04:39.924 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:39 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2291855222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T02:04:39.925 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:39 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2460423961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T02:04:39.925 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:39 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1169990120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T02:04:40.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:39 smithi172 ceph-mon[35340]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 116 B/s, 1 keys/s, 1 objects/s recovering 2024-09-16T02:04:40.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:39 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2291855222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T02:04:40.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:39 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2460423961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T02:04:40.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:39 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1169990120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T02:04:40.210 INFO:tasks.cephadm.ceph_manager.ceph:need seq 124554051643 got 124554051646 for osd.4 2024-09-16T02:04:40.211 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:40.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.317+0000 7f7393e00640 1 -- 172.21.15.103:0/3731377150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7394102360 msgr2=0x7f739410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:40.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.317+0000 7f7393e00640 1 --2- 172.21.15.103:0/3731377150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7394102360 0x7f739410e890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f738c009cd0 tx=0x7f738c02a8f0 comp rx=0 tx=0).stop 2024-09-16T02:04:40.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.317+0000 7f7393e00640 1 -- 172.21.15.103:0/3731377150 shutdown_connections 2024-09-16T02:04:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.317+0000 7f7393e00640 1 --2- 172.21.15.103:0/3731377150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7394102360 0x7f739410e890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.317+0000 7f7393e00640 1 --2- 172.21.15.103:0/3731377150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73941019c0 0x7f7394101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.317+0000 7f7393e00640 1 --2- 172.21.15.103:0/3731377150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73941079c0 0x7f7394107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.317+0000 7f7393e00640 1 -- 172.21.15.103:0/3731377150 >> 172.21.15.103:0/3731377150 conn(0x7f73940fd740 msgr2=0x7f73940ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.317+0000 7f7393e00640 1 -- 172.21.15.103:0/3731377150 shutdown_connections 2024-09-16T02:04:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.317+0000 7f7393e00640 1 -- 172.21.15.103:0/3731377150 wait complete. 2024-09-16T02:04:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.318+0000 7f7393e00640 1 Processor -- start 2024-09-16T02:04:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.318+0000 7f7393e00640 1 -- start start 2024-09-16T02:04:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7393e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73941019c0 0x7f73940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7393e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394102360 0x7f7394074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7393e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73941079c0 0x7f7394074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:40.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7393e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7394111e10 con 0x7f7394102360 2024-09-16T02:04:40.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7393e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7394111c90 con 0x7f73941079c0 2024-09-16T02:04:40.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7393e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7394111f90 con 0x7f73941019c0 2024-09-16T02:04:40.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7392000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394102360 0x7f7394074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:40.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7392a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73941019c0 0x7f73940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7392a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73941019c0 0x7f73940744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:51272/0 (socket says 172.21.15.103:51272) 2024-09-16T02:04:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7392a00640 1 -- 172.21.15.103:0/2460423961 learned_addr learned my addr 172.21.15.103:0/2460423961 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7393400640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73941079c0 0x7f7394074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7392a00640 1 -- 172.21.15.103:0/2460423961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73941079c0 msgr2=0x7f7394074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7392a00640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73941079c0 0x7f7394074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7392a00640 1 -- 172.21.15.103:0/2460423961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394102360 msgr2=0x7f7394074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:40.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7392a00640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394102360 0x7f7394074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:40.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7392a00640 1 -- 172.21.15.103:0/2460423961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73941a6730 con 0x7f73941019c0 2024-09-16T02:04:40.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.319+0000 7f7393400640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73941079c0 0x7f7394074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:40.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.320+0000 7f7392a00640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73941019c0 0x7f73940744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f738400ea60 tx=0x7f738400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:40.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.320+0000 7f7398a00640 1 -- 172.21.15.103:0/2460423961 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738400cde0 con 0x7f73941019c0 2024-09-16T02:04:40.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.320+0000 7f7398a00640 1 -- 172.21.15.103:0/2460423961 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73840035f0 con 0x7f73941019c0 2024-09-16T02:04:40.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.320+0000 7f7398a00640 1 -- 172.21.15.103:0/2460423961 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384010670 con 0x7f73941019c0 2024-09-16T02:04:40.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.320+0000 7f7393e00640 1 -- 172.21.15.103:0/2460423961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73941a6a20 con 0x7f73941019c0 2024-09-16T02:04:40.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.320+0000 7f7393e00640 1 -- 172.21.15.103:0/2460423961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73941a6f10 con 0x7f73941019c0 2024-09-16T02:04:40.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.321+0000 7f7393e00640 1 -- 172.21.15.103:0/2460423961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7394101e20 con 0x7f73941019c0 2024-09-16T02:04:40.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.325+0000 7f7398a00640 1 -- 172.21.15.103:0/2460423961 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f73840037a0 con 0x7f73941019c0 2024-09-16T02:04:40.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.326+0000 7f7398a00640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f73780a32d0 0x7f73780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:40.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.326+0000 7f7398a00640 1 -- 172.21.15.103:0/2460423961 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f73840cfe00 con 0x7f73941019c0 2024-09-16T02:04:40.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.326+0000 7f7392000640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f73780a32d0 0x7f73780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:40.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.327+0000 7f7392000640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f73780a32d0 0x7f73780a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7394075f80 tx=0x7f7380005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:40.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.330+0000 7f7398a00640 1 -- 172.21.15.103:0/2460423961 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f738409c950 con 0x7f73941019c0 2024-09-16T02:04:40.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.534+0000 7f7393e00640 1 -- 172.21.15.103:0/2460423961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f73941a7240 con 0x7f73941019c0 2024-09-16T02:04:40.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.535+0000 7f7398a00640 1 -- 172.21.15.103:0/2460423961 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f738409c250 con 0x7f73941019c0 2024-09-16T02:04:40.314 INFO:teuthology.orchestra.run.smithi103.stdout:227633266730 2024-09-16T02:04:40.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 -- 172.21.15.103:0/2460423961 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f73780a32d0 msgr2=0x7f73780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:40.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f73780a32d0 0x7f73780a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7394075f80 tx=0x7f7380005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:40.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 -- 172.21.15.103:0/2460423961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73941019c0 msgr2=0x7f73940744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:40.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73941019c0 0x7f73940744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f738400ea60 tx=0x7f738400ef20 comp rx=0 tx=0).stop 2024-09-16T02:04:40.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 -- 172.21.15.103:0/2460423961 shutdown_connections 2024-09-16T02:04:40.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f73780a32d0 0x7f73780a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:40.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73941079c0 0x7f7394074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:40.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394102360 0x7f7394074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:40.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 --2- 172.21.15.103:0/2460423961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73941019c0 0x7f73940744f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:40.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 -- 172.21.15.103:0/2460423961 >> 172.21.15.103:0/2460423961 conn(0x7f73940fd740 msgr2=0x7f73940fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:40.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 -- 172.21.15.103:0/2460423961 shutdown_connections 2024-09-16T02:04:40.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:39.537+0000 7f7393e00640 1 -- 172.21.15.103:0/2460423961 wait complete. 2024-09-16T02:04:40.879 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738449 for osd.0 2024-09-16T02:04:40.879 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:41.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 94489280576 got 94489280579 for osd.3 2024-09-16T02:04:41.508 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:41.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:41 smithi103 ceph-mon[30398]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4 op/s; 106 B/s, 0 keys/s, 1 objects/s recovering 2024-09-16T02:04:42.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:41 smithi136 ceph-mon[35323]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4 op/s; 106 B/s, 0 keys/s, 1 objects/s recovering 2024-09-16T02:04:42.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:41 smithi172 ceph-mon[35340]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4 op/s; 106 B/s, 0 keys/s, 1 objects/s recovering 2024-09-16T02:04:42.064 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266727 got 227633266730 for osd.8 2024-09-16T02:04:42.064 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:42.364 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906969 got 274877906973 for osd.10 2024-09-16T02:04:42.364 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:42.409 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:43.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:42 smithi103 ceph-mon[30398]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4 op/s; 95 B/s, 0 keys/s, 1 objects/s recovering 2024-09-16T02:04:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:42 smithi136 ceph-mon[35323]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4 op/s; 95 B/s, 0 keys/s, 1 objects/s recovering 2024-09-16T02:04:43.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:42 smithi172 ceph-mon[35340]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4 op/s; 95 B/s, 0 keys/s, 1 objects/s recovering 2024-09-16T02:04:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.110+0000 7fd91e000640 1 -- 172.21.15.103:0/138686119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918109140 msgr2=0x7fd918110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.110+0000 7fd91e000640 1 --2- 172.21.15.103:0/138686119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918109140 0x7fd918110cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd910009ca0 tx=0x7fd91002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:04:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.110+0000 7fd91e000640 1 -- 172.21.15.103:0/138686119 shutdown_connections 2024-09-16T02:04:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.110+0000 7fd91e000640 1 --2- 172.21.15.103:0/138686119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918109140 0x7fd918110cc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.110+0000 7fd91e000640 1 --2- 172.21.15.103:0/138686119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9181049c0 0x7fd918108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.110+0000 7fd91e000640 1 --2- 172.21.15.103:0/138686119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd918104010 0x7fd9181043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.110+0000 7fd91e000640 1 -- 172.21.15.103:0/138686119 >> 172.21.15.103:0/138686119 conn(0x7fd9180ffa50 msgr2=0x7fd918101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.110+0000 7fd91e000640 1 -- 172.21.15.103:0/138686119 shutdown_connections 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.110+0000 7fd91e000640 1 -- 172.21.15.103:0/138686119 wait complete. 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.111+0000 7fd91e000640 1 Processor -- start 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd91e000640 1 -- start start 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd91e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd918104010 0x7fd91819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd91e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9181049c0 0x7fd91819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd91e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918109140 0x7fd91819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd91e000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd918114170 con 0x7fd918109140 2024-09-16T02:04:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd91e000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd918113ff0 con 0x7fd918104010 2024-09-16T02:04:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd91e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd9181142f0 con 0x7fd9181049c0 2024-09-16T02:04:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd917e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918109140 0x7fd91819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd917e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918109140 0x7fd91819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33608/0 (socket says 172.21.15.103:33608) 2024-09-16T02:04:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd917e00640 1 -- 172.21.15.103:0/257189258 learned_addr learned my addr 172.21.15.103:0/257189258 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd916a00640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9181049c0 0x7fd91819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.112+0000 7fd917400640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd918104010 0x7fd91819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd917e00640 1 -- 172.21.15.103:0/257189258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9181049c0 msgr2=0x7fd91819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd917e00640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9181049c0 0x7fd91819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd917e00640 1 -- 172.21.15.103:0/257189258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd918104010 msgr2=0x7fd91819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd917e00640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd918104010 0x7fd91819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd917e00640 1 -- 172.21.15.103:0/257189258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91819f680 con 0x7fd918109140 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd917e00640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918109140 0x7fd91819efa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd910031a20 tx=0x7fd910031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd90fe00640 1 -- 172.21.15.103:0/257189258 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910003c30 con 0x7fd918109140 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd91e000640 1 -- 172.21.15.103:0/257189258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91819f910 con 0x7fd918109140 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd90fe00640 1 -- 172.21.15.103:0/257189258 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd910003dd0 con 0x7fd918109140 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd90fe00640 1 -- 172.21.15.103:0/257189258 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91002fcf0 con 0x7fd918109140 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.113+0000 7fd91e000640 1 -- 172.21.15.103:0/257189258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9181a7250 con 0x7fd918109140 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.114+0000 7fd91e000640 1 -- 172.21.15.103:0/257189258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9181043f0 con 0x7fd918109140 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.116+0000 7fd90fe00640 1 -- 172.21.15.103:0/257189258 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd910004050 con 0x7fd918109140 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.117+0000 7fd90fe00640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd8fc0a3350 0x7fd8fc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.118+0000 7fd90fe00640 1 -- 172.21.15.103:0/257189258 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fd9100f0c60 con 0x7fd918109140 2024-09-16T02:04:43.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.118+0000 7fd917400640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd8fc0a3350 0x7fd8fc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:43.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.118+0000 7fd917400640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd8fc0a3350 0x7fd8fc0a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd908005fd0 tx=0x7fd90800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:43.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.121+0000 7fd90fe00640 1 -- 172.21.15.103:0/257189258 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9100bda70 con 0x7fd918109140 2024-09-16T02:04:43.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.303+0000 7fd91e000640 1 -- 172.21.15.103:0/257189258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd91810c380 con 0x7fd918109140 2024-09-16T02:04:43.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.303+0000 7fd90fe00640 1 -- 172.21.15.103:0/257189258 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd91002fe90 con 0x7fd918109140 2024-09-16T02:04:43.306 INFO:teuthology.orchestra.run.smithi103.stdout:304942678043 2024-09-16T02:04:43.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.305+0000 7fd91e000640 1 -- 172.21.15.103:0/257189258 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd8fc0a3350 msgr2=0x7fd8fc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:43.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.305+0000 7fd91e000640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd8fc0a3350 0x7fd8fc0a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd908005fd0 tx=0x7fd90800b390 comp rx=0 tx=0).stop 2024-09-16T02:04:43.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.306+0000 7fd91e000640 1 -- 172.21.15.103:0/257189258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918109140 msgr2=0x7fd91819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:43.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.306+0000 7fd91e000640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918109140 0x7fd91819efa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd910031a20 tx=0x7fd910031a50 comp rx=0 tx=0).stop 2024-09-16T02:04:43.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.306+0000 7fd91e000640 1 -- 172.21.15.103:0/257189258 shutdown_connections 2024-09-16T02:04:43.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.306+0000 7fd91e000640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fd8fc0a3350 0x7fd8fc0a5810 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:43.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.306+0000 7fd91e000640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918109140 0x7fd91819efa0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:43.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.306+0000 7fd91e000640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9181049c0 0x7fd91819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:43.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.306+0000 7fd91e000640 1 --2- 172.21.15.103:0/257189258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd918104010 0x7fd91819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:43.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.306+0000 7fd91e000640 1 -- 172.21.15.103:0/257189258 >> 172.21.15.103:0/257189258 conn(0x7fd9180ffa50 msgr2=0x7fd918100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:43.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.306+0000 7fd91e000640 1 -- 172.21.15.103:0/257189258 shutdown_connections 2024-09-16T02:04:43.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:43.306+0000 7fd91e000640 1 -- 172.21.15.103:0/257189258 wait complete. 2024-09-16T02:04:44.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:43 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/257189258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T02:04:44.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:43 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/257189258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T02:04:44.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:43 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/257189258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T02:04:44.235 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678040 got 304942678043 for osd.11 2024-09-16T02:04:44.236 DEBUG:teuthology.parallel:result is None 2024-09-16T02:04:44.236 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T02:04:44.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:04:44.647 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:45.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:44 smithi103 ceph-mon[30398]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 93 B/s, 0 objects/s recovering 2024-09-16T02:04:45.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:44 smithi136 ceph-mon[35323]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 93 B/s, 0 objects/s recovering 2024-09-16T02:04:45.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:44 smithi172 ceph-mon[35340]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 93 B/s, 0 objects/s recovering 2024-09-16T02:04:45.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.198+0000 7f0cda200640 1 -- 172.21.15.103:0/1348664059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cd41049c0 msgr2=0x7f0cd4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:45.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.198+0000 7f0cda200640 1 --2- 172.21.15.103:0/1348664059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cd41049c0 0x7f0cd4108a10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009900 tx=0x7f0cc002a410 comp rx=0 tx=0).stop 2024-09-16T02:04:45.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.198+0000 7f0cda200640 1 -- 172.21.15.103:0/1348664059 shutdown_connections 2024-09-16T02:04:45.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.198+0000 7f0cda200640 1 --2- 172.21.15.103:0/1348664059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cd4109140 0x7f0cd4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:45.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.198+0000 7f0cda200640 1 --2- 172.21.15.103:0/1348664059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cd41049c0 0x7f0cd4108a10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:45.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.198+0000 7f0cda200640 1 --2- 172.21.15.103:0/1348664059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd4104010 0x7f0cd41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:45.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.198+0000 7f0cda200640 1 -- 172.21.15.103:0/1348664059 >> 172.21.15.103:0/1348664059 conn(0x7f0cd40ffa50 msgr2=0x7f0cd4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:45.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.199+0000 7f0cda200640 1 -- 172.21.15.103:0/1348664059 shutdown_connections 2024-09-16T02:04:45.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.199+0000 7f0cda200640 1 -- 172.21.15.103:0/1348664059 wait complete. 2024-09-16T02:04:45.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.200+0000 7f0cda200640 1 Processor -- start 2024-09-16T02:04:45.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.200+0000 7f0cda200640 1 -- start start 2024-09-16T02:04:45.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.200+0000 7f0cda200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd4104010 0x7f0cd41a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:45.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cda200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cd41049c0 0x7f0cd41a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:45.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cda200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cd4109140 0x7f0cd419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:45.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cda200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4114210 con 0x7f0cd41049c0 2024-09-16T02:04:45.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cda200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4114090 con 0x7f0cd4104010 2024-09-16T02:04:45.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cda200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4114390 con 0x7f0cd4109140 2024-09-16T02:04:45.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cd2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cd41049c0 0x7f0cd41a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:45.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cd3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cd4109140 0x7f0cd419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:45.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cd3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd4104010 0x7f0cd41a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:45.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cd3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cd4109140 0x7f0cd419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:51336/0 (socket says 172.21.15.103:51336) 2024-09-16T02:04:45.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cd2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cd41049c0 0x7f0cd41a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33626/0 (socket says 172.21.15.103:33626) 2024-09-16T02:04:45.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cd3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd4104010 0x7f0cd41a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:45616/0 (socket says 172.21.15.103:45616) 2024-09-16T02:04:45.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.201+0000 7f0cd3e00640 1 -- 172.21.15.103:0/692232034 learned_addr learned my addr 172.21.15.103:0/692232034 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:45.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0cd2a00640 1 -- 172.21.15.103:0/692232034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cd4109140 msgr2=0x7f0cd419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:45.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0cd2a00640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cd4109140 0x7f0cd419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:45.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0cd2a00640 1 -- 172.21.15.103:0/692232034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd4104010 msgr2=0x7f0cd41a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:45.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0cd2a00640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd4104010 0x7f0cd41a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:45.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0cd2a00640 1 -- 172.21.15.103:0/692232034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd419acd0 con 0x7f0cd41049c0 2024-09-16T02:04:45.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0cd2a00640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cd41049c0 0x7f0cd41a0920 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0cc002a930 tx=0x7f0cc0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:45.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0ccbe00640 1 -- 172.21.15.103:0/692232034 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc003f070 con 0x7f0cd41049c0 2024-09-16T02:04:45.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0ccbe00640 1 -- 172.21.15.103:0/692232034 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc0031c90 con 0x7f0cd41049c0 2024-09-16T02:04:45.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0cda200640 1 -- 172.21.15.103:0/692232034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd419af60 con 0x7f0cd41049c0 2024-09-16T02:04:45.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0ccbe00640 1 -- 172.21.15.103:0/692232034 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0035030 con 0x7f0cd41049c0 2024-09-16T02:04:45.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.202+0000 7f0cda200640 1 -- 172.21.15.103:0/692232034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd4076630 con 0x7f0cd41049c0 2024-09-16T02:04:45.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.204+0000 7f0cda200640 1 -- 172.21.15.103:0/692232034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd41054f0 con 0x7f0cd41049c0 2024-09-16T02:04:45.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.206+0000 7f0ccbe00640 1 -- 172.21.15.103:0/692232034 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0cc0042070 con 0x7f0cd41049c0 2024-09-16T02:04:45.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.207+0000 7f0ccbe00640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0cb80a3350 0x7f0cb80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:45.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.207+0000 7f0ccbe00640 1 -- 172.21.15.103:0/692232034 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f0cc00efd60 con 0x7f0cd41049c0 2024-09-16T02:04:45.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.207+0000 7f0cd3400640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0cb80a3350 0x7f0cb80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:45.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.208+0000 7f0cd3400640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0cb80a3350 0x7f0cb80a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009bd0 tx=0x7f0cc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:45.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.211+0000 7f0ccbe00640 1 -- 172.21.15.103:0/692232034 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cc00bc260 con 0x7f0cd41049c0 2024-09-16T02:04:45.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.403+0000 7f0cda200640 1 -- 172.21.15.103:0/692232034 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0cd4062630 con 0x7f0cb80a3350 2024-09-16T02:04:45.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.411+0000 7f0ccbe00640 1 -- 172.21.15.103:0/692232034 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258726 (secure 0 0 0) 0x7f0cd4062630 con 0x7f0cb80a3350 2024-09-16T02:04:45.412 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:45.416 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-16T02:04:45.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 -- 172.21.15.103:0/692232034 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0cb80a3350 msgr2=0x7f0cb80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:45.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0cb80a3350 0x7f0cb80a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009bd0 tx=0x7f0cc4005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:45.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 -- 172.21.15.103:0/692232034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cd41049c0 msgr2=0x7f0cd41a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:45.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cd41049c0 0x7f0cd41a0920 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0cc002a930 tx=0x7f0cc0003b90 comp rx=0 tx=0).stop 2024-09-16T02:04:45.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 -- 172.21.15.103:0/692232034 shutdown_connections 2024-09-16T02:04:45.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0cb80a3350 0x7f0cb80a5810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:45.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cd4109140 0x7f0cd419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:45.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cd41049c0 0x7f0cd41a0920 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:45.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 --2- 172.21.15.103:0/692232034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd4104010 0x7f0cd41a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:45.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 -- 172.21.15.103:0/692232034 >> 172.21.15.103:0/692232034 conn(0x7f0cd40ffa50 msgr2=0x7f0cd4100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:45.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 -- 172.21.15.103:0/692232034 shutdown_connections 2024-09-16T02:04:45.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:45.415+0000 7f0cda200640 1 -- 172.21.15.103:0/692232034 wait complete. 2024-09-16T02:04:46.179 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-16T02:04:44.464377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654762,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":475,"num_read_kb":353,"num_write":347,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181885,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369956,"kb_used_data":12300,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571868,"statfs":{"total":1151940427776,"available":1151561592832,"internally_reserved":0,"allocated":12595200,"data_stored":7034010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1127,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003430"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629731+0000","last_change":"2024-09-16T02:03:37.050873+0000","last_active":"2024-09-16T02:04:29.629731+0000","last_peered":"2024-09-16T02:04:29.629731+0000","last_clean":"2024-09-16T02:04:29.629731+0000","last_became_active":"2024-09-16T02:03:37.050202+0000","last_became_peered":"2024-09-16T02:03:37.050202+0000","last_unstale":"2024-09-16T02:04:29.629731+0000","last_undegraded":"2024-09-16T02:04:29.629731+0000","last_fullsized":"2024-09-16T02:04:29.629731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:12.971199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631138+0000","last_change":"2024-09-16T02:03:37.048034+0000","last_active":"2024-09-16T02:04:29.631138+0000","last_peered":"2024-09-16T02:04:29.631138+0000","last_clean":"2024-09-16T02:04:29.631138+0000","last_became_active":"2024-09-16T02:03:37.047216+0000","last_became_peered":"2024-09-16T02:03:37.047216+0000","last_unstale":"2024-09-16T02:04:29.631138+0000","last_undegraded":"2024-09-16T02:04:29.631138+0000","last_fullsized":"2024-09-16T02:04:29.631138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:48.580466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631092+0000","last_change":"2024-09-16T02:03:37.059287+0000","last_active":"2024-09-16T02:04:29.631092+0000","last_peered":"2024-09-16T02:04:29.631092+0000","last_clean":"2024-09-16T02:04:29.631092+0000","last_became_active":"2024-09-16T02:03:37.055270+0000","last_became_peered":"2024-09-16T02:03:37.055270+0000","last_unstale":"2024-09-16T02:04:29.631092+0000","last_undegraded":"2024-09-16T02:04:29.631092+0000","last_fullsized":"2024-09-16T02:04:29.631092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:29.430514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638028+0000","last_change":"2024-09-16T02:03:37.052473+0000","last_active":"2024-09-16T02:04:29.638028+0000","last_peered":"2024-09-16T02:04:29.638028+0000","last_clean":"2024-09-16T02:04:29.638028+0000","last_became_active":"2024-09-16T02:03:37.052015+0000","last_became_peered":"2024-09-16T02:03:37.052015+0000","last_unstale":"2024-09-16T02:04:29.638028+0000","last_undegraded":"2024-09-16T02:04:29.638028+0000","last_fullsized":"2024-09-16T02:04:29.638028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:03.964132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632294+0000","last_change":"2024-09-16T02:03:37.051516+0000","last_active":"2024-09-16T02:04:29.632294+0000","last_peered":"2024-09-16T02:04:29.632294+0000","last_clean":"2024-09-16T02:04:29.632294+0000","last_became_active":"2024-09-16T02:03:37.051201+0000","last_became_peered":"2024-09-16T02:03:37.051201+0000","last_unstale":"2024-09-16T02:04:29.632294+0000","last_undegraded":"2024-09-16T02:04:29.632294+0000","last_fullsized":"2024-09-16T02:04:29.632294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:45.327477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629772+0000","last_change":"2024-09-16T02:03:37.048912+0000","last_active":"2024-09-16T02:04:29.629772+0000","last_peered":"2024-09-16T02:04:29.629772+0000","last_clean":"2024-09-16T02:04:29.629772+0000","last_became_active":"2024-09-16T02:03:37.048577+0000","last_became_peered":"2024-09-16T02:03:37.048577+0000","last_unstale":"2024-09-16T02:04:29.629772+0000","last_undegraded":"2024-09-16T02:04:29.629772+0000","last_fullsized":"2024-09-16T02:04:29.629772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:25.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632998+0000","last_change":"2024-09-16T02:03:37.057427+0000","last_active":"2024-09-16T02:04:29.632998+0000","last_peered":"2024-09-16T02:04:29.632998+0000","last_clean":"2024-09-16T02:04:29.632998+0000","last_became_active":"2024-09-16T02:03:37.057111+0000","last_became_peered":"2024-09-16T02:03:37.057111+0000","last_unstale":"2024-09-16T02:04:29.632998+0000","last_undegraded":"2024-09-16T02:04:29.632998+0000","last_fullsized":"2024-09-16T02:04:29.632998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:44.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.717967+0000","last_change":"2024-09-16T02:03:37.051085+0000","last_active":"2024-09-16T02:04:29.717967+0000","last_peered":"2024-09-16T02:04:29.717967+0000","last_clean":"2024-09-16T02:04:29.717967+0000","last_became_active":"2024-09-16T02:03:37.050790+0000","last_became_peered":"2024-09-16T02:03:37.050790+0000","last_unstale":"2024-09-16T02:04:29.717967+0000","last_undegraded":"2024-09-16T02:04:29.717967+0000","last_fullsized":"2024-09-16T02:04:29.717967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:18.620901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637699+0000","last_change":"2024-09-16T02:03:37.054258+0000","last_active":"2024-09-16T02:04:29.637699+0000","last_peered":"2024-09-16T02:04:29.637699+0000","last_clean":"2024-09-16T02:04:29.637699+0000","last_became_active":"2024-09-16T02:03:37.053939+0000","last_became_peered":"2024-09-16T02:03:37.053939+0000","last_unstale":"2024-09-16T02:04:29.637699+0000","last_undegraded":"2024-09-16T02:04:29.637699+0000","last_fullsized":"2024-09-16T02:04:29.637699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:11.076147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.630364+0000","last_change":"2024-09-16T02:03:37.049828+0000","last_active":"2024-09-16T02:04:29.630364+0000","last_peered":"2024-09-16T02:04:29.630364+0000","last_clean":"2024-09-16T02:04:29.630364+0000","last_became_active":"2024-09-16T02:03:37.049577+0000","last_became_peered":"2024-09-16T02:03:37.049577+0000","last_unstale":"2024-09-16T02:04:29.630364+0000","last_undegraded":"2024-09-16T02:04:29.630364+0000","last_fullsized":"2024-09-16T02:04:29.630364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:37.176431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633548+0000","last_change":"2024-09-16T02:03:37.053268+0000","last_active":"2024-09-16T02:04:29.633548+0000","last_peered":"2024-09-16T02:04:29.633548+0000","last_clean":"2024-09-16T02:04:29.633548+0000","last_became_active":"2024-09-16T02:03:37.052933+0000","last_became_peered":"2024-09-16T02:03:37.052933+0000","last_unstale":"2024-09-16T02:04:29.633548+0000","last_undegraded":"2024-09-16T02:04:29.633548+0000","last_fullsized":"2024-09-16T02:04:29.633548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:32.003728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638124+0000","last_change":"2024-09-16T02:03:37.046339+0000","last_active":"2024-09-16T02:04:29.638124+0000","last_peered":"2024-09-16T02:04:29.638124+0000","last_clean":"2024-09-16T02:04:29.638124+0000","last_became_active":"2024-09-16T02:03:37.043262+0000","last_became_peered":"2024-09-16T02:03:37.043262+0000","last_unstale":"2024-09-16T02:04:29.638124+0000","last_undegraded":"2024-09-16T02:04:29.638124+0000","last_fullsized":"2024-09-16T02:04:29.638124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:58.614672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628841+0000","last_change":"2024-09-16T02:03:37.043856+0000","last_active":"2024-09-16T02:04:29.628841+0000","last_peered":"2024-09-16T02:04:29.628841+0000","last_clean":"2024-09-16T02:04:29.628841+0000","last_became_active":"2024-09-16T02:03:37.043546+0000","last_became_peered":"2024-09-16T02:03:37.043546+0000","last_unstale":"2024-09-16T02:04:29.628841+0000","last_undegraded":"2024-09-16T02:04:29.628841+0000","last_fullsized":"2024-09-16T02:04:29.628841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:57:07.213575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.630688+0000","last_change":"2024-09-16T02:03:37.048844+0000","last_active":"2024-09-16T02:04:29.630688+0000","last_peered":"2024-09-16T02:04:29.630688+0000","last_clean":"2024-09-16T02:04:29.630688+0000","last_became_active":"2024-09-16T02:03:37.048522+0000","last_became_peered":"2024-09-16T02:03:37.048522+0000","last_unstale":"2024-09-16T02:04:29.630688+0000","last_undegraded":"2024-09-16T02:04:29.630688+0000","last_fullsized":"2024-09-16T02:04:29.630688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:50.759847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.630559+0000","last_change":"2024-09-16T02:03:37.043236+0000","last_active":"2024-09-16T02:04:29.630559+0000","last_peered":"2024-09-16T02:04:29.630559+0000","last_clean":"2024-09-16T02:04:29.630559+0000","last_became_active":"2024-09-16T02:03:37.042967+0000","last_became_peered":"2024-09-16T02:03:37.042967+0000","last_unstale":"2024-09-16T02:04:29.630559+0000","last_undegraded":"2024-09-16T02:04:29.630559+0000","last_fullsized":"2024-09-16T02:04:29.630559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:02.739678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636720+0000","last_change":"2024-09-16T02:03:37.052972+0000","last_active":"2024-09-16T02:04:29.636720+0000","last_peered":"2024-09-16T02:04:29.636720+0000","last_clean":"2024-09-16T02:04:29.636720+0000","last_became_active":"2024-09-16T02:03:37.052665+0000","last_became_peered":"2024-09-16T02:03:37.052665+0000","last_unstale":"2024-09-16T02:04:29.636720+0000","last_undegraded":"2024-09-16T02:04:29.636720+0000","last_fullsized":"2024-09-16T02:04:29.636720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:59.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629466+0000","last_change":"2024-09-16T02:03:37.051203+0000","last_active":"2024-09-16T02:04:29.629466+0000","last_peered":"2024-09-16T02:04:29.629466+0000","last_clean":"2024-09-16T02:04:29.629466+0000","last_became_active":"2024-09-16T02:03:37.049828+0000","last_became_peered":"2024-09-16T02:03:37.049828+0000","last_unstale":"2024-09-16T02:04:29.629466+0000","last_undegraded":"2024-09-16T02:04:29.629466+0000","last_fullsized":"2024-09-16T02:04:29.629466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:47.566594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.719113+0000","last_change":"2024-09-16T02:03:37.056903+0000","last_active":"2024-09-16T02:04:29.719113+0000","last_peered":"2024-09-16T02:04:29.719113+0000","last_clean":"2024-09-16T02:04:29.719113+0000","last_became_active":"2024-09-16T02:03:37.056579+0000","last_became_peered":"2024-09-16T02:03:37.056579+0000","last_unstale":"2024-09-16T02:04:29.719113+0000","last_undegraded":"2024-09-16T02:04:29.719113+0000","last_fullsized":"2024-09-16T02:04:29.719113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:23.652294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631802+0000","last_change":"2024-09-16T02:03:37.054413+0000","last_active":"2024-09-16T02:04:29.631802+0000","last_peered":"2024-09-16T02:04:29.631802+0000","last_clean":"2024-09-16T02:04:29.631802+0000","last_became_active":"2024-09-16T02:03:37.054040+0000","last_became_peered":"2024-09-16T02:03:37.054040+0000","last_unstale":"2024-09-16T02:04:29.631802+0000","last_undegraded":"2024-09-16T02:04:29.631802+0000","last_fullsized":"2024-09-16T02:04:29.631802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:04.517313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629279+0000","last_change":"2024-09-16T02:03:37.053454+0000","last_active":"2024-09-16T02:04:29.629279+0000","last_peered":"2024-09-16T02:04:29.629279+0000","last_clean":"2024-09-16T02:04:29.629279+0000","last_became_active":"2024-09-16T02:03:37.053123+0000","last_became_peered":"2024-09-16T02:03:37.053123+0000","last_unstale":"2024-09-16T02:04:29.629279+0000","last_undegraded":"2024-09-16T02:04:29.629279+0000","last_fullsized":"2024-09-16T02:04:29.629279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:52.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628823+0000","last_change":"2024-09-16T02:03:37.042772+0000","last_active":"2024-09-16T02:04:29.628823+0000","last_peered":"2024-09-16T02:04:29.628823+0000","last_clean":"2024-09-16T02:04:29.628823+0000","last_became_active":"2024-09-16T02:03:37.041832+0000","last_became_peered":"2024-09-16T02:03:37.041832+0000","last_unstale":"2024-09-16T02:04:29.628823+0000","last_undegraded":"2024-09-16T02:04:29.628823+0000","last_fullsized":"2024-09-16T02:04:29.628823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:54.436291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637175+0000","last_change":"2024-09-16T02:03:37.058484+0000","last_active":"2024-09-16T02:04:29.637175+0000","last_peered":"2024-09-16T02:04:29.637175+0000","last_clean":"2024-09-16T02:04:29.637175+0000","last_became_active":"2024-09-16T02:03:37.058047+0000","last_became_peered":"2024-09-16T02:03:37.058047+0000","last_unstale":"2024-09-16T02:04:29.637175+0000","last_undegraded":"2024-09-16T02:04:29.637175+0000","last_fullsized":"2024-09-16T02:04:29.637175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:23.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718083+0000","last_change":"2024-09-16T02:03:37.058576+0000","last_active":"2024-09-16T02:04:29.718083+0000","last_peered":"2024-09-16T02:04:29.718083+0000","last_clean":"2024-09-16T02:04:29.718083+0000","last_became_active":"2024-09-16T02:03:37.057624+0000","last_became_peered":"2024-09-16T02:03:37.057624+0000","last_unstale":"2024-09-16T02:04:29.718083+0000","last_undegraded":"2024-09-16T02:04:29.718083+0000","last_fullsized":"2024-09-16T02:04:29.718083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:21.929461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638007+0000","last_change":"2024-09-16T02:03:37.050889+0000","last_active":"2024-09-16T02:04:29.638007+0000","last_peered":"2024-09-16T02:04:29.638007+0000","last_clean":"2024-09-16T02:04:29.638007+0000","last_became_active":"2024-09-16T02:03:37.050535+0000","last_became_peered":"2024-09-16T02:03:37.050535+0000","last_unstale":"2024-09-16T02:04:29.638007+0000","last_undegraded":"2024-09-16T02:04:29.638007+0000","last_fullsized":"2024-09-16T02:04:29.638007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:46.955962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718176+0000","last_change":"2024-09-16T02:03:37.053865+0000","last_active":"2024-09-16T02:04:29.718176+0000","last_peered":"2024-09-16T02:04:29.718176+0000","last_clean":"2024-09-16T02:04:29.718176+0000","last_became_active":"2024-09-16T02:03:37.053563+0000","last_became_peered":"2024-09-16T02:03:37.053563+0000","last_unstale":"2024-09-16T02:04:29.718176+0000","last_undegraded":"2024-09-16T02:04:29.718176+0000","last_fullsized":"2024-09-16T02:04:29.718176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:24.109471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632038+0000","last_change":"2024-09-16T02:03:37.054681+0000","last_active":"2024-09-16T02:04:29.632038+0000","last_peered":"2024-09-16T02:04:29.632038+0000","last_clean":"2024-09-16T02:04:29.632038+0000","last_became_active":"2024-09-16T02:03:37.054324+0000","last_became_peered":"2024-09-16T02:03:37.054324+0000","last_unstale":"2024-09-16T02:04:29.632038+0000","last_undegraded":"2024-09-16T02:04:29.632038+0000","last_fullsized":"2024-09-16T02:04:29.632038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:20.472520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629563+0000","last_change":"2024-09-16T02:03:37.047713+0000","last_active":"2024-09-16T02:04:29.629563+0000","last_peered":"2024-09-16T02:04:29.629563+0000","last_clean":"2024-09-16T02:04:29.629563+0000","last_became_active":"2024-09-16T02:03:37.047353+0000","last_became_peered":"2024-09-16T02:03:37.047353+0000","last_unstale":"2024-09-16T02:04:29.629563+0000","last_undegraded":"2024-09-16T02:04:29.629563+0000","last_fullsized":"2024-09-16T02:04:29.629563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:05.868909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628666+0000","last_change":"2024-09-16T02:03:37.038931+0000","last_active":"2024-09-16T02:04:29.628666+0000","last_peered":"2024-09-16T02:04:29.628666+0000","last_clean":"2024-09-16T02:04:29.628666+0000","last_became_active":"2024-09-16T02:03:37.037234+0000","last_became_peered":"2024-09-16T02:03:37.037234+0000","last_unstale":"2024-09-16T02:04:29.628666+0000","last_undegraded":"2024-09-16T02:04:29.628666+0000","last_fullsized":"2024-09-16T02:04:29.628666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:54.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.634327+0000","last_change":"2024-09-16T02:04:29.634327+0000","last_active":"2024-09-16T02:04:29.634327+0000","last_peered":"2024-09-16T02:04:29.634327+0000","last_clean":"2024-09-16T02:04:29.634327+0000","last_became_active":"2024-09-16T02:04:29.633710+0000","last_became_peered":"2024-09-16T02:04:29.633710+0000","last_unstale":"2024-09-16T02:04:29.634327+0000","last_undegraded":"2024-09-16T02:04:29.634327+0000","last_fullsized":"2024-09-16T02:04:29.634327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:55:03.574092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632013+0000","last_change":"2024-09-16T02:03:37.057688+0000","last_active":"2024-09-16T02:04:29.632013+0000","last_peered":"2024-09-16T02:04:29.632013+0000","last_clean":"2024-09-16T02:04:29.632013+0000","last_became_active":"2024-09-16T02:03:37.057347+0000","last_became_peered":"2024-09-16T02:03:37.057347+0000","last_unstale":"2024-09-16T02:04:29.632013+0000","last_undegraded":"2024-09-16T02:04:29.632013+0000","last_fullsized":"2024-09-16T02:04:29.632013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:30.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.640455+0000","last_change":"2024-09-16T02:04:29.640455+0000","last_active":"2024-09-16T02:04:29.640455+0000","last_peered":"2024-09-16T02:04:29.640455+0000","last_clean":"2024-09-16T02:04:29.640455+0000","last_became_active":"2024-09-16T02:04:29.640083+0000","last_became_peered":"2024-09-16T02:04:29.640083+0000","last_unstale":"2024-09-16T02:04:29.640455+0000","last_undegraded":"2024-09-16T02:04:29.640455+0000","last_fullsized":"2024-09-16T02:04:29.640455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:46.191484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629179+0000","last_change":"2024-09-16T02:03:37.043596+0000","last_active":"2024-09-16T02:04:29.629179+0000","last_peered":"2024-09-16T02:04:29.629179+0000","last_clean":"2024-09-16T02:04:29.629179+0000","last_became_active":"2024-09-16T02:03:37.043261+0000","last_became_peered":"2024-09-16T02:03:37.043261+0000","last_unstale":"2024-09-16T02:04:29.629179+0000","last_undegraded":"2024-09-16T02:04:29.629179+0000","last_fullsized":"2024-09-16T02:04:29.629179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:03.833126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632123+0000","last_change":"2024-09-16T02:03:37.043018+0000","last_active":"2024-09-16T02:04:29.632123+0000","last_peered":"2024-09-16T02:04:29.632123+0000","last_clean":"2024-09-16T02:04:29.632123+0000","last_became_active":"2024-09-16T02:03:37.042437+0000","last_became_peered":"2024-09-16T02:03:37.042437+0000","last_unstale":"2024-09-16T02:04:29.632123+0000","last_undegraded":"2024-09-16T02:04:29.632123+0000","last_fullsized":"2024-09-16T02:04:29.632123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.193852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":97,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:40.657648+0000","last_change":"2024-09-16T02:04:29.645481+0000","last_active":"2024-09-16T02:04:40.657648+0000","last_peered":"2024-09-16T02:04:40.657648+0000","last_clean":"2024-09-16T02:04:40.657648+0000","last_became_active":"2024-09-16T02:04:29.640089+0000","last_became_peered":"2024-09-16T02:04:29.640089+0000","last_unstale":"2024-09-16T02:04:40.657648+0000","last_undegraded":"2024-09-16T02:04:40.657648+0000","last_fullsized":"2024-09-16T02:04:40.657648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:02.431443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638369+0000","last_change":"2024-09-16T02:03:37.052664+0000","last_active":"2024-09-16T02:04:29.638369+0000","last_peered":"2024-09-16T02:04:29.638369+0000","last_clean":"2024-09-16T02:04:29.638369+0000","last_became_active":"2024-09-16T02:03:37.051310+0000","last_became_peered":"2024-09-16T02:03:37.051310+0000","last_unstale":"2024-09-16T02:04:29.638369+0000","last_undegraded":"2024-09-16T02:04:29.638369+0000","last_fullsized":"2024-09-16T02:04:29.638369+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:57.979053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.654377+0000","last_change":"2024-09-16T02:04:29.654377+0000","last_active":"2024-09-16T02:04:29.654377+0000","last_peered":"2024-09-16T02:04:29.654377+0000","last_clean":"2024-09-16T02:04:29.654377+0000","last_became_active":"2024-09-16T02:04:29.635187+0000","last_became_peered":"2024-09-16T02:04:29.635187+0000","last_unstale":"2024-09-16T02:04:29.654377+0000","last_undegraded":"2024-09-16T02:04:29.654377+0000","last_fullsized":"2024-09-16T02:04:29.654377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:28.474841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,6],"acting":[11,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628421+0000","last_change":"2024-09-16T02:03:34.745325+0000","last_active":"2024-09-16T02:04:29.628421+0000","last_peered":"2024-09-16T02:04:29.628421+0000","last_clean":"2024-09-16T02:04:29.628421+0000","last_became_active":"2024-09-16T02:03:34.744730+0000","last_became_peered":"2024-09-16T02:03:34.744730+0000","last_unstale":"2024-09-16T02:04:29.628421+0000","last_undegraded":"2024-09-16T02:04:29.628421+0000","last_fullsized":"2024-09-16T02:04:29.628421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718030+0000","last_change":"2024-09-16T02:03:37.058973+0000","last_active":"2024-09-16T02:04:29.718030+0000","last_peered":"2024-09-16T02:04:29.718030+0000","last_clean":"2024-09-16T02:04:29.718030+0000","last_became_active":"2024-09-16T02:03:37.058685+0000","last_became_peered":"2024-09-16T02:03:37.058685+0000","last_unstale":"2024-09-16T02:04:29.718030+0000","last_undegraded":"2024-09-16T02:04:29.718030+0000","last_fullsized":"2024-09-16T02:04:29.718030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:46.030440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638192+0000","last_change":"2024-09-16T02:03:34.737842+0000","last_active":"2024-09-16T02:04:29.638192+0000","last_peered":"2024-09-16T02:04:29.638192+0000","last_clean":"2024-09-16T02:04:29.638192+0000","last_became_active":"2024-09-16T02:03:34.737391+0000","last_became_peered":"2024-09-16T02:03:34.737391+0000","last_unstale":"2024-09-16T02:04:29.638192+0000","last_undegraded":"2024-09-16T02:04:29.638192+0000","last_fullsized":"2024-09-16T02:04:29.638192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:39.214180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718894+0000","last_change":"2024-09-16T02:03:37.049705+0000","last_active":"2024-09-16T02:04:29.718894+0000","last_peered":"2024-09-16T02:04:29.718894+0000","last_clean":"2024-09-16T02:04:29.718894+0000","last_became_active":"2024-09-16T02:03:37.049392+0000","last_became_peered":"2024-09-16T02:03:37.049392+0000","last_unstale":"2024-09-16T02:04:29.718894+0000","last_undegraded":"2024-09-16T02:04:29.718894+0000","last_fullsized":"2024-09-16T02:04:29.718894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.932391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":80,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:41.658339+0000","last_change":"2024-09-16T02:03:34.742268+0000","last_active":"2024-09-16T02:04:41.658339+0000","last_peered":"2024-09-16T02:04:41.658339+0000","last_clean":"2024-09-16T02:04:41.658339+0000","last_became_active":"2024-09-16T02:03:34.741934+0000","last_became_peered":"2024-09-16T02:03:34.741934+0000","last_unstale":"2024-09-16T02:04:41.658339+0000","last_undegraded":"2024-09-16T02:04:41.658339+0000","last_fullsized":"2024-09-16T02:04:41.658339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:30.301552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637511+0000","last_change":"2024-09-16T02:03:37.059119+0000","last_active":"2024-09-16T02:04:29.637511+0000","last_peered":"2024-09-16T02:04:29.637511+0000","last_clean":"2024-09-16T02:04:29.637511+0000","last_became_active":"2024-09-16T02:03:37.058738+0000","last_became_peered":"2024-09-16T02:03:37.058738+0000","last_unstale":"2024-09-16T02:04:29.637511+0000","last_undegraded":"2024-09-16T02:04:29.637511+0000","last_fullsized":"2024-09-16T02:04:29.637511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.496053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.642379+0000","last_change":"2024-09-16T02:04:29.642379+0000","last_active":"2024-09-16T02:04:29.642379+0000","last_peered":"2024-09-16T02:04:29.642379+0000","last_clean":"2024-09-16T02:04:29.642379+0000","last_became_active":"2024-09-16T02:04:29.638587+0000","last_became_peered":"2024-09-16T02:04:29.638587+0000","last_unstale":"2024-09-16T02:04:29.642379+0000","last_undegraded":"2024-09-16T02:04:29.642379+0000","last_fullsized":"2024-09-16T02:04:29.642379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:50.268223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631290+0000","last_change":"2024-09-16T02:03:37.050349+0000","last_active":"2024-09-16T02:04:29.631290+0000","last_peered":"2024-09-16T02:04:29.631290+0000","last_clean":"2024-09-16T02:04:29.631290+0000","last_became_active":"2024-09-16T02:03:37.049978+0000","last_became_peered":"2024-09-16T02:03:37.049978+0000","last_unstale":"2024-09-16T02:04:29.631290+0000","last_undegraded":"2024-09-16T02:04:29.631290+0000","last_fullsized":"2024-09-16T02:04:29.631290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:45.119683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637417+0000","last_change":"2024-09-16T02:03:34.740246+0000","last_active":"2024-09-16T02:04:29.637417+0000","last_peered":"2024-09-16T02:04:29.637417+0000","last_clean":"2024-09-16T02:04:29.637417+0000","last_became_active":"2024-09-16T02:03:34.739908+0000","last_became_peered":"2024-09-16T02:03:34.739908+0000","last_unstale":"2024-09-16T02:04:29.637417+0000","last_undegraded":"2024-09-16T02:04:29.637417+0000","last_fullsized":"2024-09-16T02:04:29.637417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:35.299733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636954+0000","last_change":"2024-09-16T02:03:37.056562+0000","last_active":"2024-09-16T02:04:29.636954+0000","last_peered":"2024-09-16T02:04:29.636954+0000","last_clean":"2024-09-16T02:04:29.636954+0000","last_became_active":"2024-09-16T02:03:37.056226+0000","last_became_peered":"2024-09-16T02:03:37.056226+0000","last_unstale":"2024-09-16T02:04:29.636954+0000","last_undegraded":"2024-09-16T02:04:29.636954+0000","last_fullsized":"2024-09-16T02:04:29.636954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.741538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'28","reported_seq":169,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:42.659738+0000","last_change":"2024-09-16T02:03:34.736403+0000","last_active":"2024-09-16T02:04:42.659738+0000","last_peered":"2024-09-16T02:04:42.659738+0000","last_clean":"2024-09-16T02:04:42.659738+0000","last_became_active":"2024-09-16T02:03:34.736074+0000","last_became_peered":"2024-09-16T02:03:34.736074+0000","last_unstale":"2024-09-16T02:04:42.659738+0000","last_undegraded":"2024-09-16T02:04:42.659738+0000","last_fullsized":"2024-09-16T02:04:42.659738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:34.769862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":119,"num_read_kb":117,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632824+0000","last_change":"2024-09-16T02:03:37.046803+0000","last_active":"2024-09-16T02:04:29.632824+0000","last_peered":"2024-09-16T02:04:29.632824+0000","last_clean":"2024-09-16T02:04:29.632824+0000","last_became_active":"2024-09-16T02:03:37.046459+0000","last_became_peered":"2024-09-16T02:03:37.046459+0000","last_unstale":"2024-09-16T02:04:29.632824+0000","last_undegraded":"2024-09-16T02:04:29.632824+0000","last_fullsized":"2024-09-16T02:04:29.632824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:55:38.460795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632119+0000","last_change":"2024-09-16T02:03:34.739016+0000","last_active":"2024-09-16T02:04:29.632119+0000","last_peered":"2024-09-16T02:04:29.632119+0000","last_clean":"2024-09-16T02:04:29.632119+0000","last_became_active":"2024-09-16T02:03:34.738583+0000","last_became_peered":"2024-09-16T02:03:34.738583+0000","last_unstale":"2024-09-16T02:04:29.632119+0000","last_undegraded":"2024-09-16T02:04:29.632119+0000","last_fullsized":"2024-09-16T02:04:29.632119+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:08.255832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628509+0000","last_change":"2024-09-16T02:03:37.053353+0000","last_active":"2024-09-16T02:04:29.628509+0000","last_peered":"2024-09-16T02:04:29.628509+0000","last_clean":"2024-09-16T02:04:29.628509+0000","last_became_active":"2024-09-16T02:03:37.051140+0000","last_became_peered":"2024-09-16T02:03:37.051140+0000","last_unstale":"2024-09-16T02:04:29.628509+0000","last_undegraded":"2024-09-16T02:04:29.628509+0000","last_fullsized":"2024-09-16T02:04:29.628509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:34.636338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636210+0000","last_change":"2024-09-16T02:03:34.742111+0000","last_active":"2024-09-16T02:04:29.636210+0000","last_peered":"2024-09-16T02:04:29.636210+0000","last_clean":"2024-09-16T02:04:29.636210+0000","last_became_active":"2024-09-16T02:03:34.741816+0000","last_became_peered":"2024-09-16T02:03:34.741816+0000","last_unstale":"2024-09-16T02:04:29.636210+0000","last_undegraded":"2024-09-16T02:04:29.636210+0000","last_fullsized":"2024-09-16T02:04:29.636210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:36.861995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632976+0000","last_change":"2024-09-16T02:03:37.047152+0000","last_active":"2024-09-16T02:04:29.632976+0000","last_peered":"2024-09-16T02:04:29.632976+0000","last_clean":"2024-09-16T02:04:29.632976+0000","last_became_active":"2024-09-16T02:03:37.046675+0000","last_became_peered":"2024-09-16T02:03:37.046675+0000","last_unstale":"2024-09-16T02:04:29.632976+0000","last_undegraded":"2024-09-16T02:04:29.632976+0000","last_fullsized":"2024-09-16T02:04:29.632976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:58.236519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":84,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:41.182933+0000","last_change":"2024-09-16T02:03:34.738487+0000","last_active":"2024-09-16T02:04:41.182933+0000","last_peered":"2024-09-16T02:04:41.182933+0000","last_clean":"2024-09-16T02:04:41.182933+0000","last_became_active":"2024-09-16T02:03:34.737061+0000","last_became_peered":"2024-09-16T02:03:34.737061+0000","last_unstale":"2024-09-16T02:04:41.182933+0000","last_undegraded":"2024-09-16T02:04:41.182933+0000","last_fullsized":"2024-09-16T02:04:41.182933+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:24.231640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632969+0000","last_change":"2024-09-16T02:03:37.044260+0000","last_active":"2024-09-16T02:04:29.632969+0000","last_peered":"2024-09-16T02:04:29.632969+0000","last_clean":"2024-09-16T02:04:29.632969+0000","last_became_active":"2024-09-16T02:03:37.043932+0000","last_became_peered":"2024-09-16T02:03:37.043932+0000","last_unstale":"2024-09-16T02:04:29.632969+0000","last_undegraded":"2024-09-16T02:04:29.632969+0000","last_fullsized":"2024-09-16T02:04:29.632969+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:55.718155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.651235+0000","last_change":"2024-09-16T02:04:29.651235+0000","last_active":"2024-09-16T02:04:29.651235+0000","last_peered":"2024-09-16T02:04:29.651235+0000","last_clean":"2024-09-16T02:04:29.651235+0000","last_became_active":"2024-09-16T02:04:29.633999+0000","last_became_peered":"2024-09-16T02:04:29.633999+0000","last_unstale":"2024-09-16T02:04:29.651235+0000","last_undegraded":"2024-09-16T02:04:29.651235+0000","last_fullsized":"2024-09-16T02:04:29.651235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.616990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636190+0000","last_change":"2024-09-16T02:03:37.053040+0000","last_active":"2024-09-16T02:04:29.636190+0000","last_peered":"2024-09-16T02:04:29.636190+0000","last_clean":"2024-09-16T02:04:29.636190+0000","last_became_active":"2024-09-16T02:03:37.052185+0000","last_became_peered":"2024-09-16T02:03:37.052185+0000","last_unstale":"2024-09-16T02:04:29.636190+0000","last_undegraded":"2024-09-16T02:04:29.636190+0000","last_fullsized":"2024-09-16T02:04:29.636190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:27.676678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.651376+0000","last_change":"2024-09-16T02:04:29.651376+0000","last_active":"2024-09-16T02:04:29.651376+0000","last_peered":"2024-09-16T02:04:29.651376+0000","last_clean":"2024-09-16T02:04:29.651376+0000","last_became_active":"2024-09-16T02:04:29.640260+0000","last_became_peered":"2024-09-16T02:04:29.640260+0000","last_unstale":"2024-09-16T02:04:29.651376+0000","last_undegraded":"2024-09-16T02:04:29.651376+0000","last_fullsized":"2024-09-16T02:04:29.651376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:39.426183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629483+0000","last_change":"2024-09-16T02:03:37.049420+0000","last_active":"2024-09-16T02:04:29.629483+0000","last_peered":"2024-09-16T02:04:29.629483+0000","last_clean":"2024-09-16T02:04:29.629483+0000","last_became_active":"2024-09-16T02:03:37.047671+0000","last_became_peered":"2024-09-16T02:03:37.047671+0000","last_unstale":"2024-09-16T02:04:29.629483+0000","last_undegraded":"2024-09-16T02:04:29.629483+0000","last_fullsized":"2024-09-16T02:04:29.629483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:23.023791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628718+0000","last_change":"2024-09-16T02:03:34.738516+0000","last_active":"2024-09-16T02:04:29.628718+0000","last_peered":"2024-09-16T02:04:29.628718+0000","last_clean":"2024-09-16T02:04:29.628718+0000","last_became_active":"2024-09-16T02:03:34.737701+0000","last_became_peered":"2024-09-16T02:03:34.737701+0000","last_unstale":"2024-09-16T02:04:29.628718+0000","last_undegraded":"2024-09-16T02:04:29.628718+0000","last_fullsized":"2024-09-16T02:04:29.628718+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:42.101865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633435+0000","last_change":"2024-09-16T02:03:37.046820+0000","last_active":"2024-09-16T02:04:29.633435+0000","last_peered":"2024-09-16T02:04:29.633435+0000","last_clean":"2024-09-16T02:04:29.633435+0000","last_became_active":"2024-09-16T02:03:37.046082+0000","last_became_peered":"2024-09-16T02:03:37.046082+0000","last_unstale":"2024-09-16T02:04:29.633435+0000","last_undegraded":"2024-09-16T02:04:29.633435+0000","last_fullsized":"2024-09-16T02:04:29.633435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:21.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637516+0000","last_change":"2024-09-16T02:03:34.741908+0000","last_active":"2024-09-16T02:04:29.637516+0000","last_peered":"2024-09-16T02:04:29.637516+0000","last_clean":"2024-09-16T02:04:29.637516+0000","last_became_active":"2024-09-16T02:03:34.739826+0000","last_became_peered":"2024-09-16T02:03:34.739826+0000","last_unstale":"2024-09-16T02:04:29.637516+0000","last_undegraded":"2024-09-16T02:04:29.637516+0000","last_fullsized":"2024-09-16T02:04:29.637516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:04.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633229+0000","last_change":"2024-09-16T02:03:37.056459+0000","last_active":"2024-09-16T02:04:29.633229+0000","last_peered":"2024-09-16T02:04:29.633229+0000","last_clean":"2024-09-16T02:04:29.633229+0000","last_became_active":"2024-09-16T02:03:37.055902+0000","last_became_peered":"2024-09-16T02:03:37.055902+0000","last_unstale":"2024-09-16T02:04:29.633229+0000","last_undegraded":"2024-09-16T02:04:29.633229+0000","last_fullsized":"2024-09-16T02:04:29.633229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:48:30.815654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632874+0000","last_change":"2024-09-16T02:03:34.741207+0000","last_active":"2024-09-16T02:04:29.632874+0000","last_peered":"2024-09-16T02:04:29.632874+0000","last_clean":"2024-09-16T02:04:29.632874+0000","last_became_active":"2024-09-16T02:03:34.740523+0000","last_became_peered":"2024-09-16T02:03:34.740523+0000","last_unstale":"2024-09-16T02:04:29.632874+0000","last_undegraded":"2024-09-16T02:04:29.632874+0000","last_fullsized":"2024-09-16T02:04:29.632874+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:36.199332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633942+0000","last_change":"2024-09-16T02:03:37.052792+0000","last_active":"2024-09-16T02:04:29.633942+0000","last_peered":"2024-09-16T02:04:29.633942+0000","last_clean":"2024-09-16T02:04:29.633942+0000","last_became_active":"2024-09-16T02:03:37.052301+0000","last_became_peered":"2024-09-16T02:03:37.052301+0000","last_unstale":"2024-09-16T02:04:29.633942+0000","last_undegraded":"2024-09-16T02:04:29.633942+0000","last_fullsized":"2024-09-16T02:04:29.633942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:55.197481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":333,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633949+0000","last_change":"2024-09-16T02:02:21.095893+0000","last_active":"2024-09-16T02:04:29.633949+0000","last_peered":"2024-09-16T02:04:29.633949+0000","last_clean":"2024-09-16T02:04:29.633949+0000","last_became_active":"2024-09-16T02:02:21.095395+0000","last_became_peered":"2024-09-16T02:02:21.095395+0000","last_unstale":"2024-09-16T02:04:29.633949+0000","last_undegraded":"2024-09-16T02:04:29.633949+0000","last_fullsized":"2024-09-16T02:04:29.633949+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_clean_scrub_stamp":"2024-09-16T01:58:50.954254+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:40.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638205+0000","last_change":"2024-09-16T02:03:34.746154+0000","last_active":"2024-09-16T02:04:29.638205+0000","last_peered":"2024-09-16T02:04:29.638205+0000","last_clean":"2024-09-16T02:04:29.638205+0000","last_became_active":"2024-09-16T02:03:34.745646+0000","last_became_peered":"2024-09-16T02:03:34.745646+0000","last_unstale":"2024-09-16T02:04:29.638205+0000","last_undegraded":"2024-09-16T02:04:29.638205+0000","last_fullsized":"2024-09-16T02:04:29.638205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:06.790000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718478+0000","last_change":"2024-09-16T02:03:37.056638+0000","last_active":"2024-09-16T02:04:29.718478+0000","last_peered":"2024-09-16T02:04:29.718478+0000","last_clean":"2024-09-16T02:04:29.718478+0000","last_became_active":"2024-09-16T02:03:37.056043+0000","last_became_peered":"2024-09-16T02:03:37.056043+0000","last_unstale":"2024-09-16T02:04:29.718478+0000","last_undegraded":"2024-09-16T02:04:29.718478+0000","last_fullsized":"2024-09-16T02:04:29.718478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:35:48.353380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.647682+0000","last_change":"2024-09-16T02:04:29.647682+0000","last_active":"2024-09-16T02:04:29.647682+0000","last_peered":"2024-09-16T02:04:29.647682+0000","last_clean":"2024-09-16T02:04:29.647682+0000","last_became_active":"2024-09-16T02:04:29.640065+0000","last_became_peered":"2024-09-16T02:04:29.640065+0000","last_unstale":"2024-09-16T02:04:29.647682+0000","last_undegraded":"2024-09-16T02:04:29.647682+0000","last_fullsized":"2024-09-16T02:04:29.647682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:53.765121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628419+0000","last_change":"2024-09-16T02:03:37.050895+0000","last_active":"2024-09-16T02:04:29.628419+0000","last_peered":"2024-09-16T02:04:29.628419+0000","last_clean":"2024-09-16T02:04:29.628419+0000","last_became_active":"2024-09-16T02:03:37.050559+0000","last_became_peered":"2024-09-16T02:03:37.050559+0000","last_unstale":"2024-09-16T02:04:29.628419+0000","last_undegraded":"2024-09-16T02:04:29.628419+0000","last_fullsized":"2024-09-16T02:04:29.628419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:20.067954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":79,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:39.179782+0000","last_change":"2024-09-16T02:03:34.740286+0000","last_active":"2024-09-16T02:04:39.179782+0000","last_peered":"2024-09-16T02:04:39.179782+0000","last_clean":"2024-09-16T02:04:39.179782+0000","last_became_active":"2024-09-16T02:03:34.739892+0000","last_became_peered":"2024-09-16T02:03:34.739892+0000","last_unstale":"2024-09-16T02:04:39.179782+0000","last_undegraded":"2024-09-16T02:04:39.179782+0000","last_fullsized":"2024-09-16T02:04:39.179782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:59:14.809991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638092+0000","last_change":"2024-09-16T02:03:37.042207+0000","last_active":"2024-09-16T02:04:29.638092+0000","last_peered":"2024-09-16T02:04:29.638092+0000","last_clean":"2024-09-16T02:04:29.638092+0000","last_became_active":"2024-09-16T02:03:37.041283+0000","last_became_peered":"2024-09-16T02:03:37.041283+0000","last_unstale":"2024-09-16T02:04:29.638092+0000","last_undegraded":"2024-09-16T02:04:29.638092+0000","last_fullsized":"2024-09-16T02:04:29.638092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:45.569721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633269+0000","last_change":"2024-09-16T02:03:34.737834+0000","last_active":"2024-09-16T02:04:29.633269+0000","last_peered":"2024-09-16T02:04:29.633269+0000","last_clean":"2024-09-16T02:04:29.633269+0000","last_became_active":"2024-09-16T02:03:34.736706+0000","last_became_peered":"2024-09-16T02:03:34.736706+0000","last_unstale":"2024-09-16T02:04:29.633269+0000","last_undegraded":"2024-09-16T02:04:29.633269+0000","last_fullsized":"2024-09-16T02:04:29.633269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:27.037351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.630069+0000","last_change":"2024-09-16T02:03:37.060888+0000","last_active":"2024-09-16T02:04:29.630069+0000","last_peered":"2024-09-16T02:04:29.630069+0000","last_clean":"2024-09-16T02:04:29.630069+0000","last_became_active":"2024-09-16T02:03:37.060458+0000","last_became_peered":"2024-09-16T02:03:37.060458+0000","last_unstale":"2024-09-16T02:04:29.630069+0000","last_undegraded":"2024-09-16T02:04:29.630069+0000","last_fullsized":"2024-09-16T02:04:29.630069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:37.425376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.719071+0000","last_change":"2024-09-16T02:03:34.742298+0000","last_active":"2024-09-16T02:04:29.719071+0000","last_peered":"2024-09-16T02:04:29.719071+0000","last_clean":"2024-09-16T02:04:29.719071+0000","last_became_active":"2024-09-16T02:03:34.741960+0000","last_became_peered":"2024-09-16T02:03:34.741960+0000","last_unstale":"2024-09-16T02:04:29.719071+0000","last_undegraded":"2024-09-16T02:04:29.719071+0000","last_fullsized":"2024-09-16T02:04:29.719071+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:12.832207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629969+0000","last_change":"2024-09-16T02:03:37.060553+0000","last_active":"2024-09-16T02:04:29.629969+0000","last_peered":"2024-09-16T02:04:29.629969+0000","last_clean":"2024-09-16T02:04:29.629969+0000","last_became_active":"2024-09-16T02:03:37.059961+0000","last_became_peered":"2024-09-16T02:03:37.059961+0000","last_unstale":"2024-09-16T02:04:29.629969+0000","last_undegraded":"2024-09-16T02:04:29.629969+0000","last_fullsized":"2024-09-16T02:04:29.629969+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:22.904190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633669+0000","last_change":"2024-09-16T02:03:34.747828+0000","last_active":"2024-09-16T02:04:29.633669+0000","last_peered":"2024-09-16T02:04:29.633669+0000","last_clean":"2024-09-16T02:04:29.633669+0000","last_became_active":"2024-09-16T02:03:34.747548+0000","last_became_peered":"2024-09-16T02:03:34.747548+0000","last_unstale":"2024-09-16T02:04:29.633669+0000","last_undegraded":"2024-09-16T02:04:29.633669+0000","last_fullsized":"2024-09-16T02:04:29.633669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:49.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628783+0000","last_change":"2024-09-16T02:03:37.051477+0000","last_active":"2024-09-16T02:04:29.628783+0000","last_peered":"2024-09-16T02:04:29.628783+0000","last_clean":"2024-09-16T02:04:29.628783+0000","last_became_active":"2024-09-16T02:03:37.048063+0000","last_became_peered":"2024-09-16T02:03:37.048063+0000","last_unstale":"2024-09-16T02:04:29.628783+0000","last_undegraded":"2024-09-16T02:04:29.628783+0000","last_fullsized":"2024-09-16T02:04:29.628783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:15.973972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.719147+0000","last_change":"2024-09-16T02:03:34.742935+0000","last_active":"2024-09-16T02:04:29.719147+0000","last_peered":"2024-09-16T02:04:29.719147+0000","last_clean":"2024-09-16T02:04:29.719147+0000","last_became_active":"2024-09-16T02:03:34.742611+0000","last_became_peered":"2024-09-16T02:03:34.742611+0000","last_unstale":"2024-09-16T02:04:29.719147+0000","last_undegraded":"2024-09-16T02:04:29.719147+0000","last_fullsized":"2024-09-16T02:04:29.719147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:36.695220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718255+0000","last_change":"2024-09-16T02:03:37.054926+0000","last_active":"2024-09-16T02:04:29.718255+0000","last_peered":"2024-09-16T02:04:29.718255+0000","last_clean":"2024-09-16T02:04:29.718255+0000","last_became_active":"2024-09-16T02:03:37.054483+0000","last_became_peered":"2024-09-16T02:03:37.054483+0000","last_unstale":"2024-09-16T02:04:29.718255+0000","last_undegraded":"2024-09-16T02:04:29.718255+0000","last_fullsized":"2024-09-16T02:04:29.718255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638845+0000","last_change":"2024-09-16T02:03:34.739611+0000","last_active":"2024-09-16T02:04:29.638845+0000","last_peered":"2024-09-16T02:04:29.638845+0000","last_clean":"2024-09-16T02:04:29.638845+0000","last_became_active":"2024-09-16T02:03:34.738090+0000","last_became_peered":"2024-09-16T02:03:34.738090+0000","last_unstale":"2024-09-16T02:04:29.638845+0000","last_undegraded":"2024-09-16T02:04:29.638845+0000","last_fullsized":"2024-09-16T02:04:29.638845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:47.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631158+0000","last_change":"2024-09-16T02:03:37.053020+0000","last_active":"2024-09-16T02:04:29.631158+0000","last_peered":"2024-09-16T02:04:29.631158+0000","last_clean":"2024-09-16T02:04:29.631158+0000","last_became_active":"2024-09-16T02:03:37.052365+0000","last_became_peered":"2024-09-16T02:03:37.052365+0000","last_unstale":"2024-09-16T02:04:29.631158+0000","last_undegraded":"2024-09-16T02:04:29.631158+0000","last_fullsized":"2024-09-16T02:04:29.631158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:54.965028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636913+0000","last_change":"2024-09-16T02:03:34.740726+0000","last_active":"2024-09-16T02:04:29.636913+0000","last_peered":"2024-09-16T02:04:29.636913+0000","last_clean":"2024-09-16T02:04:29.636913+0000","last_became_active":"2024-09-16T02:03:34.740402+0000","last_became_peered":"2024-09-16T02:03:34.740402+0000","last_unstale":"2024-09-16T02:04:29.636913+0000","last_undegraded":"2024-09-16T02:04:29.636913+0000","last_fullsized":"2024-09-16T02:04:29.636913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:05.761646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631417+0000","last_change":"2024-09-16T02:03:37.044862+0000","last_active":"2024-09-16T02:04:29.631417+0000","last_peered":"2024-09-16T02:04:29.631417+0000","last_clean":"2024-09-16T02:04:29.631417+0000","last_became_active":"2024-09-16T02:03:37.044406+0000","last_became_peered":"2024-09-16T02:03:37.044406+0000","last_unstale":"2024-09-16T02:04:29.631417+0000","last_undegraded":"2024-09-16T02:04:29.631417+0000","last_fullsized":"2024-09-16T02:04:29.631417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:03.326380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718201+0000","last_change":"2024-09-16T02:03:34.745505+0000","last_active":"2024-09-16T02:04:29.718201+0000","last_peered":"2024-09-16T02:04:29.718201+0000","last_clean":"2024-09-16T02:04:29.718201+0000","last_became_active":"2024-09-16T02:03:34.745151+0000","last_became_peered":"2024-09-16T02:03:34.745151+0000","last_unstale":"2024-09-16T02:04:29.718201+0000","last_undegraded":"2024-09-16T02:04:29.718201+0000","last_fullsized":"2024-09-16T02:04:29.718201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:57.401176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629299+0000","last_change":"2024-09-16T02:03:37.046692+0000","last_active":"2024-09-16T02:04:29.629299+0000","last_peered":"2024-09-16T02:04:29.629299+0000","last_clean":"2024-09-16T02:04:29.629299+0000","last_became_active":"2024-09-16T02:03:37.046302+0000","last_became_peered":"2024-09-16T02:03:37.046302+0000","last_unstale":"2024-09-16T02:04:29.629299+0000","last_undegraded":"2024-09-16T02:04:29.629299+0000","last_fullsized":"2024-09-16T02:04:29.629299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:47.727431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632142+0000","last_change":"2024-09-16T02:03:34.739847+0000","last_active":"2024-09-16T02:04:29.632142+0000","last_peered":"2024-09-16T02:04:29.632142+0000","last_clean":"2024-09-16T02:04:29.632142+0000","last_became_active":"2024-09-16T02:03:34.739514+0000","last_became_peered":"2024-09-16T02:03:34.739514+0000","last_unstale":"2024-09-16T02:04:29.632142+0000","last_undegraded":"2024-09-16T02:04:29.632142+0000","last_fullsized":"2024-09-16T02:04:29.632142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:44.617127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718141+0000","last_change":"2024-09-16T02:03:37.054910+0000","last_active":"2024-09-16T02:04:29.718141+0000","last_peered":"2024-09-16T02:04:29.718141+0000","last_clean":"2024-09-16T02:04:29.718141+0000","last_became_active":"2024-09-16T02:03:37.053852+0000","last_became_peered":"2024-09-16T02:03:37.053852+0000","last_unstale":"2024-09-16T02:04:29.718141+0000","last_undegraded":"2024-09-16T02:04:29.718141+0000","last_fullsized":"2024-09-16T02:04:29.718141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:29.042154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632406+0000","last_change":"2024-09-16T02:03:34.743027+0000","last_active":"2024-09-16T02:04:29.632406+0000","last_peered":"2024-09-16T02:04:29.632406+0000","last_clean":"2024-09-16T02:04:29.632406+0000","last_became_active":"2024-09-16T02:03:34.742658+0000","last_became_peered":"2024-09-16T02:03:34.742658+0000","last_unstale":"2024-09-16T02:04:29.632406+0000","last_undegraded":"2024-09-16T02:04:29.632406+0000","last_fullsized":"2024-09-16T02:04:29.632406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:36.779568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633061+0000","last_change":"2024-09-16T02:03:37.052783+0000","last_active":"2024-09-16T02:04:29.633061+0000","last_peered":"2024-09-16T02:04:29.633061+0000","last_clean":"2024-09-16T02:04:29.633061+0000","last_became_active":"2024-09-16T02:03:37.052409+0000","last_became_peered":"2024-09-16T02:03:37.052409+0000","last_unstale":"2024-09-16T02:04:29.633061+0000","last_undegraded":"2024-09-16T02:04:29.633061+0000","last_fullsized":"2024-09-16T02:04:29.633061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:52.667379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637314+0000","last_change":"2024-09-16T02:03:37.053645+0000","last_active":"2024-09-16T02:04:29.637314+0000","last_peered":"2024-09-16T02:04:29.637314+0000","last_clean":"2024-09-16T02:04:29.637314+0000","last_became_active":"2024-09-16T02:03:37.053340+0000","last_became_peered":"2024-09-16T02:03:37.053340+0000","last_unstale":"2024-09-16T02:04:29.637314+0000","last_undegraded":"2024-09-16T02:04:29.637314+0000","last_fullsized":"2024-09-16T02:04:29.637314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:59.827249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631820+0000","last_change":"2024-09-16T02:03:34.744240+0000","last_active":"2024-09-16T02:04:29.631820+0000","last_peered":"2024-09-16T02:04:29.631820+0000","last_clean":"2024-09-16T02:04:29.631820+0000","last_became_active":"2024-09-16T02:03:34.743898+0000","last_became_peered":"2024-09-16T02:03:34.743898+0000","last_unstale":"2024-09-16T02:04:29.631820+0000","last_undegraded":"2024-09-16T02:04:29.631820+0000","last_fullsized":"2024-09-16T02:04:29.631820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:37.820529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636830+0000","last_change":"2024-09-16T02:03:37.055769+0000","last_active":"2024-09-16T02:04:29.636830+0000","last_peered":"2024-09-16T02:04:29.636830+0000","last_clean":"2024-09-16T02:04:29.636830+0000","last_became_active":"2024-09-16T02:03:37.054458+0000","last_became_peered":"2024-09-16T02:03:37.054458+0000","last_unstale":"2024-09-16T02:04:29.636830+0000","last_undegraded":"2024-09-16T02:04:29.636830+0000","last_fullsized":"2024-09-16T02:04:29.636830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:30.218712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.647534+0000","last_change":"2024-09-16T02:04:29.647534+0000","last_active":"2024-09-16T02:04:29.647534+0000","last_peered":"2024-09-16T02:04:29.647534+0000","last_clean":"2024-09-16T02:04:29.647534+0000","last_became_active":"2024-09-16T02:04:29.640317+0000","last_became_peered":"2024-09-16T02:04:29.640317+0000","last_unstale":"2024-09-16T02:04:29.647534+0000","last_undegraded":"2024-09-16T02:04:29.647534+0000","last_fullsized":"2024-09-16T02:04:29.647534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:14:33.490508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,6],"acting":[8,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631287+0000","last_change":"2024-09-16T02:03:37.054370+0000","last_active":"2024-09-16T02:04:29.631287+0000","last_peered":"2024-09-16T02:04:29.631287+0000","last_clean":"2024-09-16T02:04:29.631287+0000","last_became_active":"2024-09-16T02:03:37.054051+0000","last_became_peered":"2024-09-16T02:03:37.054051+0000","last_unstale":"2024-09-16T02:04:29.631287+0000","last_undegraded":"2024-09-16T02:04:29.631287+0000","last_fullsized":"2024-09-16T02:04:29.631287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:21:42.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718654+0000","last_change":"2024-09-16T02:03:34.739696+0000","last_active":"2024-09-16T02:04:29.718654+0000","last_peered":"2024-09-16T02:04:29.718654+0000","last_clean":"2024-09-16T02:04:29.718654+0000","last_became_active":"2024-09-16T02:03:34.738867+0000","last_became_peered":"2024-09-16T02:03:34.738867+0000","last_unstale":"2024-09-16T02:04:29.718654+0000","last_undegraded":"2024-09-16T02:04:29.718654+0000","last_fullsized":"2024-09-16T02:04:29.718654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:30.753970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718201+0000","last_change":"2024-09-16T02:03:37.058118+0000","last_active":"2024-09-16T02:04:29.718201+0000","last_peered":"2024-09-16T02:04:29.718201+0000","last_clean":"2024-09-16T02:04:29.718201+0000","last_became_active":"2024-09-16T02:03:37.057391+0000","last_became_peered":"2024-09-16T02:03:37.057391+0000","last_unstale":"2024-09-16T02:04:29.718201+0000","last_undegraded":"2024-09-16T02:04:29.718201+0000","last_fullsized":"2024-09-16T02:04:29.718201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:39.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.719186+0000","last_change":"2024-09-16T02:03:34.742184+0000","last_active":"2024-09-16T02:04:29.719186+0000","last_peered":"2024-09-16T02:04:29.719186+0000","last_clean":"2024-09-16T02:04:29.719186+0000","last_became_active":"2024-09-16T02:03:34.741838+0000","last_became_peered":"2024-09-16T02:03:34.741838+0000","last_unstale":"2024-09-16T02:04:29.719186+0000","last_undegraded":"2024-09-16T02:04:29.719186+0000","last_fullsized":"2024-09-16T02:04:29.719186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:57.338380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":349,"num_read_kb":244,"num_write":164,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1149,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":423171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1564672,"data_stored":1074697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037796,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":884736,"data_stored":423295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266731,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":876544,"data_stored":423103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495663,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32636,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712516,"statfs":{"total":95995035648,"available":95961616384,"internally_reserved":0,"allocated":1478656,"data_stored":1015540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":422684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888121,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1486848,"data_stored":1014809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051647,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":884736,"data_stored":424529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280581,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":888832,"data_stored":422995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476809,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":950272,"data_stored":482969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607632,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":937984,"data_stored":482263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":892928,"data_stored":423955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":59786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:04:46.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T02:04:46.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:46 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:46.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:46 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:46.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:46 smithi103 ceph-mon[30398]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:46.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:46 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:04:46.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:46 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:04:46.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:46 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:46.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:46 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:46.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:46 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:46.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:46 smithi136 ceph-mon[35323]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:46.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:46 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:04:46.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:46 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:04:46.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:46 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:46.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:46 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:46.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:46 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:46.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:46 smithi172 ceph-mon[35340]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:46.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:46 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:04:46.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:46 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:04:46.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:46 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:04:46.554 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:47.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.104+0000 7fe7df200640 1 -- 172.21.15.103:0/1548033520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e01019c0 msgr2=0x7fe7e0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:47.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.104+0000 7fe7df200640 1 --2- 172.21.15.103:0/1548033520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e01019c0 0x7fe7e0101e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009930 tx=0x7fe7cc02a420 comp rx=0 tx=0).stop 2024-09-16T02:04:47.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.104+0000 7fe7df200640 1 -- 172.21.15.103:0/1548033520 shutdown_connections 2024-09-16T02:04:47.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.104+0000 7fe7df200640 1 --2- 172.21.15.103:0/1548033520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e0102360 0x7fe7e010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:47.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.104+0000 7fe7df200640 1 --2- 172.21.15.103:0/1548033520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e01019c0 0x7fe7e0101e20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:47.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.104+0000 7fe7df200640 1 --2- 172.21.15.103:0/1548033520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01079c0 0x7fe7e0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:47.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.104+0000 7fe7df200640 1 -- 172.21.15.103:0/1548033520 >> 172.21.15.103:0/1548033520 conn(0x7fe7e00fd740 msgr2=0x7fe7e00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:47.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.104+0000 7fe7df200640 1 -- 172.21.15.103:0/1548033520 shutdown_connections 2024-09-16T02:04:47.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.104+0000 7fe7df200640 1 -- 172.21.15.103:0/1548033520 wait complete. 2024-09-16T02:04:47.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7df200640 1 Processor -- start 2024-09-16T02:04:47.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7df200640 1 -- start start 2024-09-16T02:04:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7df200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e01019c0 0x7fe7e019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7df200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e0102360 0x7fe7e019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7df200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e01079c0 0x7fe7e019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7df200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0111e10 con 0x7fe7e01019c0 2024-09-16T02:04:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7df200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0111c90 con 0x7fe7e0102360 2024-09-16T02:04:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7df200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0111f90 con 0x7fe7e01079c0 2024-09-16T02:04:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7dde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e01019c0 0x7fe7e019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7dde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e01019c0 0x7fe7e019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33640/0 (socket says 172.21.15.103:33640) 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.105+0000 7fe7dde00640 1 -- 172.21.15.103:0/294421660 learned_addr learned my addr 172.21.15.103:0/294421660 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7dd400640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e0102360 0x7fe7e019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7de800640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e01079c0 0x7fe7e019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7dde00640 1 -- 172.21.15.103:0/294421660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e01079c0 msgr2=0x7fe7e019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7dde00640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e01079c0 0x7fe7e019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7dde00640 1 -- 172.21.15.103:0/294421660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e0102360 msgr2=0x7fe7e019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7dde00640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e0102360 0x7fe7e019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7dde00640 1 -- 172.21.15.103:0/294421660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e019f520 con 0x7fe7e01019c0 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7dde00640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e01019c0 0x7fe7e019a570 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000d950 tx=0x7fe7d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7d6a00640 1 -- 172.21.15.103:0/294421660 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0017070 con 0x7fe7e01019c0 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7d6a00640 1 -- 172.21.15.103:0/294421660 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d0009c30 con 0x7fe7e01019c0 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7df200640 1 -- 172.21.15.103:0/294421660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e019f810 con 0x7fe7e01019c0 2024-09-16T02:04:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.106+0000 7fe7d6a00640 1 -- 172.21.15.103:0/294421660 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000f6b0 con 0x7fe7e01019c0 2024-09-16T02:04:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.107+0000 7fe7df200640 1 -- 172.21.15.103:0/294421660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e01a70f0 con 0x7fe7e01019c0 2024-09-16T02:04:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.108+0000 7fe7df200640 1 -- 172.21.15.103:0/294421660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e0101e20 con 0x7fe7e01019c0 2024-09-16T02:04:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.110+0000 7fe7d6a00640 1 -- 172.21.15.103:0/294421660 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe7d00109d0 con 0x7fe7e01019c0 2024-09-16T02:04:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.111+0000 7fe7d6a00640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe7c40a32d0 0x7fe7c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.111+0000 7fe7d6a00640 1 -- 172.21.15.103:0/294421660 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fe7d00cf5f0 con 0x7fe7e01019c0 2024-09-16T02:04:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.111+0000 7fe7dd400640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe7c40a32d0 0x7fe7c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.112+0000 7fe7dd400640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe7c40a32d0 0x7fe7c40a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc006010 tx=0x7fe7cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:47.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.115+0000 7fe7d6a00640 1 -- 172.21.15.103:0/294421660 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7d0009de0 con 0x7fe7e01019c0 2024-09-16T02:04:47.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:47 smithi103 ceph-mon[30398]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T02:04:47.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.297+0000 7fe7df200640 1 -- 172.21.15.103:0/294421660 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7e00fecf0 con 0x7fe7c40a32d0 2024-09-16T02:04:47.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.304+0000 7fe7d6a00640 1 -- 172.21.15.103:0/294421660 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258724 (secure 0 0 0) 0x7fe7e00fecf0 con 0x7fe7c40a32d0 2024-09-16T02:04:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:47.310 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-16T02:04:47.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 -- 172.21.15.103:0/294421660 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe7c40a32d0 msgr2=0x7fe7c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:47.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe7c40a32d0 0x7fe7c40a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc006010 tx=0x7fe7cc000bc0 comp rx=0 tx=0).stop 2024-09-16T02:04:47.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 -- 172.21.15.103:0/294421660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e01019c0 msgr2=0x7fe7e019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:47.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e01019c0 0x7fe7e019a570 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000d950 tx=0x7fe7d000de10 comp rx=0 tx=0).stop 2024-09-16T02:04:47.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 -- 172.21.15.103:0/294421660 shutdown_connections 2024-09-16T02:04:47.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe7c40a32d0 0x7fe7c40a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:47.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e01079c0 0x7fe7e019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:47.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e0102360 0x7fe7e019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:47.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 --2- 172.21.15.103:0/294421660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e01019c0 0x7fe7e019a570 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:47.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 -- 172.21.15.103:0/294421660 >> 172.21.15.103:0/294421660 conn(0x7fe7e00fd740 msgr2=0x7fe7e010d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:47.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 -- 172.21.15.103:0/294421660 shutdown_connections 2024-09-16T02:04:47.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:47.308+0000 7fe7df200640 1 -- 172.21.15.103:0/294421660 wait complete. 2024-09-16T02:04:47.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:47 smithi136 ceph-mon[35323]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T02:04:47.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:47 smithi172 ceph-mon[35340]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T02:04:48.035 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-16T02:04:46.464982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654762,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":480,"num_read_kb":358,"num_write":347,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181885,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369956,"kb_used_data":12300,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571868,"statfs":{"total":1151940427776,"available":1151561592832,"internally_reserved":0,"allocated":12595200,"data_stored":7034010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003547"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629731+0000","last_change":"2024-09-16T02:03:37.050873+0000","last_active":"2024-09-16T02:04:29.629731+0000","last_peered":"2024-09-16T02:04:29.629731+0000","last_clean":"2024-09-16T02:04:29.629731+0000","last_became_active":"2024-09-16T02:03:37.050202+0000","last_became_peered":"2024-09-16T02:03:37.050202+0000","last_unstale":"2024-09-16T02:04:29.629731+0000","last_undegraded":"2024-09-16T02:04:29.629731+0000","last_fullsized":"2024-09-16T02:04:29.629731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:12.971199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631138+0000","last_change":"2024-09-16T02:03:37.048034+0000","last_active":"2024-09-16T02:04:29.631138+0000","last_peered":"2024-09-16T02:04:29.631138+0000","last_clean":"2024-09-16T02:04:29.631138+0000","last_became_active":"2024-09-16T02:03:37.047216+0000","last_became_peered":"2024-09-16T02:03:37.047216+0000","last_unstale":"2024-09-16T02:04:29.631138+0000","last_undegraded":"2024-09-16T02:04:29.631138+0000","last_fullsized":"2024-09-16T02:04:29.631138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:48.580466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631092+0000","last_change":"2024-09-16T02:03:37.059287+0000","last_active":"2024-09-16T02:04:29.631092+0000","last_peered":"2024-09-16T02:04:29.631092+0000","last_clean":"2024-09-16T02:04:29.631092+0000","last_became_active":"2024-09-16T02:03:37.055270+0000","last_became_peered":"2024-09-16T02:03:37.055270+0000","last_unstale":"2024-09-16T02:04:29.631092+0000","last_undegraded":"2024-09-16T02:04:29.631092+0000","last_fullsized":"2024-09-16T02:04:29.631092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:29.430514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638028+0000","last_change":"2024-09-16T02:03:37.052473+0000","last_active":"2024-09-16T02:04:29.638028+0000","last_peered":"2024-09-16T02:04:29.638028+0000","last_clean":"2024-09-16T02:04:29.638028+0000","last_became_active":"2024-09-16T02:03:37.052015+0000","last_became_peered":"2024-09-16T02:03:37.052015+0000","last_unstale":"2024-09-16T02:04:29.638028+0000","last_undegraded":"2024-09-16T02:04:29.638028+0000","last_fullsized":"2024-09-16T02:04:29.638028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:03.964132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632294+0000","last_change":"2024-09-16T02:03:37.051516+0000","last_active":"2024-09-16T02:04:29.632294+0000","last_peered":"2024-09-16T02:04:29.632294+0000","last_clean":"2024-09-16T02:04:29.632294+0000","last_became_active":"2024-09-16T02:03:37.051201+0000","last_became_peered":"2024-09-16T02:03:37.051201+0000","last_unstale":"2024-09-16T02:04:29.632294+0000","last_undegraded":"2024-09-16T02:04:29.632294+0000","last_fullsized":"2024-09-16T02:04:29.632294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:45.327477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629772+0000","last_change":"2024-09-16T02:03:37.048912+0000","last_active":"2024-09-16T02:04:29.629772+0000","last_peered":"2024-09-16T02:04:29.629772+0000","last_clean":"2024-09-16T02:04:29.629772+0000","last_became_active":"2024-09-16T02:03:37.048577+0000","last_became_peered":"2024-09-16T02:03:37.048577+0000","last_unstale":"2024-09-16T02:04:29.629772+0000","last_undegraded":"2024-09-16T02:04:29.629772+0000","last_fullsized":"2024-09-16T02:04:29.629772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:25.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632998+0000","last_change":"2024-09-16T02:03:37.057427+0000","last_active":"2024-09-16T02:04:29.632998+0000","last_peered":"2024-09-16T02:04:29.632998+0000","last_clean":"2024-09-16T02:04:29.632998+0000","last_became_active":"2024-09-16T02:03:37.057111+0000","last_became_peered":"2024-09-16T02:03:37.057111+0000","last_unstale":"2024-09-16T02:04:29.632998+0000","last_undegraded":"2024-09-16T02:04:29.632998+0000","last_fullsized":"2024-09-16T02:04:29.632998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:44.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.717967+0000","last_change":"2024-09-16T02:03:37.051085+0000","last_active":"2024-09-16T02:04:29.717967+0000","last_peered":"2024-09-16T02:04:29.717967+0000","last_clean":"2024-09-16T02:04:29.717967+0000","last_became_active":"2024-09-16T02:03:37.050790+0000","last_became_peered":"2024-09-16T02:03:37.050790+0000","last_unstale":"2024-09-16T02:04:29.717967+0000","last_undegraded":"2024-09-16T02:04:29.717967+0000","last_fullsized":"2024-09-16T02:04:29.717967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:18.620901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637699+0000","last_change":"2024-09-16T02:03:37.054258+0000","last_active":"2024-09-16T02:04:29.637699+0000","last_peered":"2024-09-16T02:04:29.637699+0000","last_clean":"2024-09-16T02:04:29.637699+0000","last_became_active":"2024-09-16T02:03:37.053939+0000","last_became_peered":"2024-09-16T02:03:37.053939+0000","last_unstale":"2024-09-16T02:04:29.637699+0000","last_undegraded":"2024-09-16T02:04:29.637699+0000","last_fullsized":"2024-09-16T02:04:29.637699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:11.076147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.630364+0000","last_change":"2024-09-16T02:03:37.049828+0000","last_active":"2024-09-16T02:04:29.630364+0000","last_peered":"2024-09-16T02:04:29.630364+0000","last_clean":"2024-09-16T02:04:29.630364+0000","last_became_active":"2024-09-16T02:03:37.049577+0000","last_became_peered":"2024-09-16T02:03:37.049577+0000","last_unstale":"2024-09-16T02:04:29.630364+0000","last_undegraded":"2024-09-16T02:04:29.630364+0000","last_fullsized":"2024-09-16T02:04:29.630364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:37.176431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633548+0000","last_change":"2024-09-16T02:03:37.053268+0000","last_active":"2024-09-16T02:04:29.633548+0000","last_peered":"2024-09-16T02:04:29.633548+0000","last_clean":"2024-09-16T02:04:29.633548+0000","last_became_active":"2024-09-16T02:03:37.052933+0000","last_became_peered":"2024-09-16T02:03:37.052933+0000","last_unstale":"2024-09-16T02:04:29.633548+0000","last_undegraded":"2024-09-16T02:04:29.633548+0000","last_fullsized":"2024-09-16T02:04:29.633548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:32.003728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638124+0000","last_change":"2024-09-16T02:03:37.046339+0000","last_active":"2024-09-16T02:04:29.638124+0000","last_peered":"2024-09-16T02:04:29.638124+0000","last_clean":"2024-09-16T02:04:29.638124+0000","last_became_active":"2024-09-16T02:03:37.043262+0000","last_became_peered":"2024-09-16T02:03:37.043262+0000","last_unstale":"2024-09-16T02:04:29.638124+0000","last_undegraded":"2024-09-16T02:04:29.638124+0000","last_fullsized":"2024-09-16T02:04:29.638124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:58.614672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628841+0000","last_change":"2024-09-16T02:03:37.043856+0000","last_active":"2024-09-16T02:04:29.628841+0000","last_peered":"2024-09-16T02:04:29.628841+0000","last_clean":"2024-09-16T02:04:29.628841+0000","last_became_active":"2024-09-16T02:03:37.043546+0000","last_became_peered":"2024-09-16T02:03:37.043546+0000","last_unstale":"2024-09-16T02:04:29.628841+0000","last_undegraded":"2024-09-16T02:04:29.628841+0000","last_fullsized":"2024-09-16T02:04:29.628841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:57:07.213575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.630688+0000","last_change":"2024-09-16T02:03:37.048844+0000","last_active":"2024-09-16T02:04:29.630688+0000","last_peered":"2024-09-16T02:04:29.630688+0000","last_clean":"2024-09-16T02:04:29.630688+0000","last_became_active":"2024-09-16T02:03:37.048522+0000","last_became_peered":"2024-09-16T02:03:37.048522+0000","last_unstale":"2024-09-16T02:04:29.630688+0000","last_undegraded":"2024-09-16T02:04:29.630688+0000","last_fullsized":"2024-09-16T02:04:29.630688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:50.759847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.630559+0000","last_change":"2024-09-16T02:03:37.043236+0000","last_active":"2024-09-16T02:04:29.630559+0000","last_peered":"2024-09-16T02:04:29.630559+0000","last_clean":"2024-09-16T02:04:29.630559+0000","last_became_active":"2024-09-16T02:03:37.042967+0000","last_became_peered":"2024-09-16T02:03:37.042967+0000","last_unstale":"2024-09-16T02:04:29.630559+0000","last_undegraded":"2024-09-16T02:04:29.630559+0000","last_fullsized":"2024-09-16T02:04:29.630559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:02.739678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636720+0000","last_change":"2024-09-16T02:03:37.052972+0000","last_active":"2024-09-16T02:04:29.636720+0000","last_peered":"2024-09-16T02:04:29.636720+0000","last_clean":"2024-09-16T02:04:29.636720+0000","last_became_active":"2024-09-16T02:03:37.052665+0000","last_became_peered":"2024-09-16T02:03:37.052665+0000","last_unstale":"2024-09-16T02:04:29.636720+0000","last_undegraded":"2024-09-16T02:04:29.636720+0000","last_fullsized":"2024-09-16T02:04:29.636720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:59.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629466+0000","last_change":"2024-09-16T02:03:37.051203+0000","last_active":"2024-09-16T02:04:29.629466+0000","last_peered":"2024-09-16T02:04:29.629466+0000","last_clean":"2024-09-16T02:04:29.629466+0000","last_became_active":"2024-09-16T02:03:37.049828+0000","last_became_peered":"2024-09-16T02:03:37.049828+0000","last_unstale":"2024-09-16T02:04:29.629466+0000","last_undegraded":"2024-09-16T02:04:29.629466+0000","last_fullsized":"2024-09-16T02:04:29.629466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:47.566594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.719113+0000","last_change":"2024-09-16T02:03:37.056903+0000","last_active":"2024-09-16T02:04:29.719113+0000","last_peered":"2024-09-16T02:04:29.719113+0000","last_clean":"2024-09-16T02:04:29.719113+0000","last_became_active":"2024-09-16T02:03:37.056579+0000","last_became_peered":"2024-09-16T02:03:37.056579+0000","last_unstale":"2024-09-16T02:04:29.719113+0000","last_undegraded":"2024-09-16T02:04:29.719113+0000","last_fullsized":"2024-09-16T02:04:29.719113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:23.652294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631802+0000","last_change":"2024-09-16T02:03:37.054413+0000","last_active":"2024-09-16T02:04:29.631802+0000","last_peered":"2024-09-16T02:04:29.631802+0000","last_clean":"2024-09-16T02:04:29.631802+0000","last_became_active":"2024-09-16T02:03:37.054040+0000","last_became_peered":"2024-09-16T02:03:37.054040+0000","last_unstale":"2024-09-16T02:04:29.631802+0000","last_undegraded":"2024-09-16T02:04:29.631802+0000","last_fullsized":"2024-09-16T02:04:29.631802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:04.517313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629279+0000","last_change":"2024-09-16T02:03:37.053454+0000","last_active":"2024-09-16T02:04:29.629279+0000","last_peered":"2024-09-16T02:04:29.629279+0000","last_clean":"2024-09-16T02:04:29.629279+0000","last_became_active":"2024-09-16T02:03:37.053123+0000","last_became_peered":"2024-09-16T02:03:37.053123+0000","last_unstale":"2024-09-16T02:04:29.629279+0000","last_undegraded":"2024-09-16T02:04:29.629279+0000","last_fullsized":"2024-09-16T02:04:29.629279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:52.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628823+0000","last_change":"2024-09-16T02:03:37.042772+0000","last_active":"2024-09-16T02:04:29.628823+0000","last_peered":"2024-09-16T02:04:29.628823+0000","last_clean":"2024-09-16T02:04:29.628823+0000","last_became_active":"2024-09-16T02:03:37.041832+0000","last_became_peered":"2024-09-16T02:03:37.041832+0000","last_unstale":"2024-09-16T02:04:29.628823+0000","last_undegraded":"2024-09-16T02:04:29.628823+0000","last_fullsized":"2024-09-16T02:04:29.628823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:54.436291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637175+0000","last_change":"2024-09-16T02:03:37.058484+0000","last_active":"2024-09-16T02:04:29.637175+0000","last_peered":"2024-09-16T02:04:29.637175+0000","last_clean":"2024-09-16T02:04:29.637175+0000","last_became_active":"2024-09-16T02:03:37.058047+0000","last_became_peered":"2024-09-16T02:03:37.058047+0000","last_unstale":"2024-09-16T02:04:29.637175+0000","last_undegraded":"2024-09-16T02:04:29.637175+0000","last_fullsized":"2024-09-16T02:04:29.637175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:23.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718083+0000","last_change":"2024-09-16T02:03:37.058576+0000","last_active":"2024-09-16T02:04:29.718083+0000","last_peered":"2024-09-16T02:04:29.718083+0000","last_clean":"2024-09-16T02:04:29.718083+0000","last_became_active":"2024-09-16T02:03:37.057624+0000","last_became_peered":"2024-09-16T02:03:37.057624+0000","last_unstale":"2024-09-16T02:04:29.718083+0000","last_undegraded":"2024-09-16T02:04:29.718083+0000","last_fullsized":"2024-09-16T02:04:29.718083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:21.929461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638007+0000","last_change":"2024-09-16T02:03:37.050889+0000","last_active":"2024-09-16T02:04:29.638007+0000","last_peered":"2024-09-16T02:04:29.638007+0000","last_clean":"2024-09-16T02:04:29.638007+0000","last_became_active":"2024-09-16T02:03:37.050535+0000","last_became_peered":"2024-09-16T02:03:37.050535+0000","last_unstale":"2024-09-16T02:04:29.638007+0000","last_undegraded":"2024-09-16T02:04:29.638007+0000","last_fullsized":"2024-09-16T02:04:29.638007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:46.955962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718176+0000","last_change":"2024-09-16T02:03:37.053865+0000","last_active":"2024-09-16T02:04:29.718176+0000","last_peered":"2024-09-16T02:04:29.718176+0000","last_clean":"2024-09-16T02:04:29.718176+0000","last_became_active":"2024-09-16T02:03:37.053563+0000","last_became_peered":"2024-09-16T02:03:37.053563+0000","last_unstale":"2024-09-16T02:04:29.718176+0000","last_undegraded":"2024-09-16T02:04:29.718176+0000","last_fullsized":"2024-09-16T02:04:29.718176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:24.109471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632038+0000","last_change":"2024-09-16T02:03:37.054681+0000","last_active":"2024-09-16T02:04:29.632038+0000","last_peered":"2024-09-16T02:04:29.632038+0000","last_clean":"2024-09-16T02:04:29.632038+0000","last_became_active":"2024-09-16T02:03:37.054324+0000","last_became_peered":"2024-09-16T02:03:37.054324+0000","last_unstale":"2024-09-16T02:04:29.632038+0000","last_undegraded":"2024-09-16T02:04:29.632038+0000","last_fullsized":"2024-09-16T02:04:29.632038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:20.472520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629563+0000","last_change":"2024-09-16T02:03:37.047713+0000","last_active":"2024-09-16T02:04:29.629563+0000","last_peered":"2024-09-16T02:04:29.629563+0000","last_clean":"2024-09-16T02:04:29.629563+0000","last_became_active":"2024-09-16T02:03:37.047353+0000","last_became_peered":"2024-09-16T02:03:37.047353+0000","last_unstale":"2024-09-16T02:04:29.629563+0000","last_undegraded":"2024-09-16T02:04:29.629563+0000","last_fullsized":"2024-09-16T02:04:29.629563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:05.868909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628666+0000","last_change":"2024-09-16T02:03:37.038931+0000","last_active":"2024-09-16T02:04:29.628666+0000","last_peered":"2024-09-16T02:04:29.628666+0000","last_clean":"2024-09-16T02:04:29.628666+0000","last_became_active":"2024-09-16T02:03:37.037234+0000","last_became_peered":"2024-09-16T02:03:37.037234+0000","last_unstale":"2024-09-16T02:04:29.628666+0000","last_undegraded":"2024-09-16T02:04:29.628666+0000","last_fullsized":"2024-09-16T02:04:29.628666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:54.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.634327+0000","last_change":"2024-09-16T02:04:29.634327+0000","last_active":"2024-09-16T02:04:29.634327+0000","last_peered":"2024-09-16T02:04:29.634327+0000","last_clean":"2024-09-16T02:04:29.634327+0000","last_became_active":"2024-09-16T02:04:29.633710+0000","last_became_peered":"2024-09-16T02:04:29.633710+0000","last_unstale":"2024-09-16T02:04:29.634327+0000","last_undegraded":"2024-09-16T02:04:29.634327+0000","last_fullsized":"2024-09-16T02:04:29.634327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:55:03.574092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632013+0000","last_change":"2024-09-16T02:03:37.057688+0000","last_active":"2024-09-16T02:04:29.632013+0000","last_peered":"2024-09-16T02:04:29.632013+0000","last_clean":"2024-09-16T02:04:29.632013+0000","last_became_active":"2024-09-16T02:03:37.057347+0000","last_became_peered":"2024-09-16T02:03:37.057347+0000","last_unstale":"2024-09-16T02:04:29.632013+0000","last_undegraded":"2024-09-16T02:04:29.632013+0000","last_fullsized":"2024-09-16T02:04:29.632013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:30.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.640455+0000","last_change":"2024-09-16T02:04:29.640455+0000","last_active":"2024-09-16T02:04:29.640455+0000","last_peered":"2024-09-16T02:04:29.640455+0000","last_clean":"2024-09-16T02:04:29.640455+0000","last_became_active":"2024-09-16T02:04:29.640083+0000","last_became_peered":"2024-09-16T02:04:29.640083+0000","last_unstale":"2024-09-16T02:04:29.640455+0000","last_undegraded":"2024-09-16T02:04:29.640455+0000","last_fullsized":"2024-09-16T02:04:29.640455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:46.191484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629179+0000","last_change":"2024-09-16T02:03:37.043596+0000","last_active":"2024-09-16T02:04:29.629179+0000","last_peered":"2024-09-16T02:04:29.629179+0000","last_clean":"2024-09-16T02:04:29.629179+0000","last_became_active":"2024-09-16T02:03:37.043261+0000","last_became_peered":"2024-09-16T02:03:37.043261+0000","last_unstale":"2024-09-16T02:04:29.629179+0000","last_undegraded":"2024-09-16T02:04:29.629179+0000","last_fullsized":"2024-09-16T02:04:29.629179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:03.833126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632123+0000","last_change":"2024-09-16T02:03:37.043018+0000","last_active":"2024-09-16T02:04:29.632123+0000","last_peered":"2024-09-16T02:04:29.632123+0000","last_clean":"2024-09-16T02:04:29.632123+0000","last_became_active":"2024-09-16T02:03:37.042437+0000","last_became_peered":"2024-09-16T02:03:37.042437+0000","last_unstale":"2024-09-16T02:04:29.632123+0000","last_undegraded":"2024-09-16T02:04:29.632123+0000","last_fullsized":"2024-09-16T02:04:29.632123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.193852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":97,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:40.657648+0000","last_change":"2024-09-16T02:04:29.645481+0000","last_active":"2024-09-16T02:04:40.657648+0000","last_peered":"2024-09-16T02:04:40.657648+0000","last_clean":"2024-09-16T02:04:40.657648+0000","last_became_active":"2024-09-16T02:04:29.640089+0000","last_became_peered":"2024-09-16T02:04:29.640089+0000","last_unstale":"2024-09-16T02:04:40.657648+0000","last_undegraded":"2024-09-16T02:04:40.657648+0000","last_fullsized":"2024-09-16T02:04:40.657648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:02.431443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638369+0000","last_change":"2024-09-16T02:03:37.052664+0000","last_active":"2024-09-16T02:04:29.638369+0000","last_peered":"2024-09-16T02:04:29.638369+0000","last_clean":"2024-09-16T02:04:29.638369+0000","last_became_active":"2024-09-16T02:03:37.051310+0000","last_became_peered":"2024-09-16T02:03:37.051310+0000","last_unstale":"2024-09-16T02:04:29.638369+0000","last_undegraded":"2024-09-16T02:04:29.638369+0000","last_fullsized":"2024-09-16T02:04:29.638369+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:57.979053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.654377+0000","last_change":"2024-09-16T02:04:29.654377+0000","last_active":"2024-09-16T02:04:29.654377+0000","last_peered":"2024-09-16T02:04:29.654377+0000","last_clean":"2024-09-16T02:04:29.654377+0000","last_became_active":"2024-09-16T02:04:29.635187+0000","last_became_peered":"2024-09-16T02:04:29.635187+0000","last_unstale":"2024-09-16T02:04:29.654377+0000","last_undegraded":"2024-09-16T02:04:29.654377+0000","last_fullsized":"2024-09-16T02:04:29.654377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:28.474841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,6],"acting":[11,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628421+0000","last_change":"2024-09-16T02:03:34.745325+0000","last_active":"2024-09-16T02:04:29.628421+0000","last_peered":"2024-09-16T02:04:29.628421+0000","last_clean":"2024-09-16T02:04:29.628421+0000","last_became_active":"2024-09-16T02:03:34.744730+0000","last_became_peered":"2024-09-16T02:03:34.744730+0000","last_unstale":"2024-09-16T02:04:29.628421+0000","last_undegraded":"2024-09-16T02:04:29.628421+0000","last_fullsized":"2024-09-16T02:04:29.628421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718030+0000","last_change":"2024-09-16T02:03:37.058973+0000","last_active":"2024-09-16T02:04:29.718030+0000","last_peered":"2024-09-16T02:04:29.718030+0000","last_clean":"2024-09-16T02:04:29.718030+0000","last_became_active":"2024-09-16T02:03:37.058685+0000","last_became_peered":"2024-09-16T02:03:37.058685+0000","last_unstale":"2024-09-16T02:04:29.718030+0000","last_undegraded":"2024-09-16T02:04:29.718030+0000","last_fullsized":"2024-09-16T02:04:29.718030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:46.030440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638192+0000","last_change":"2024-09-16T02:03:34.737842+0000","last_active":"2024-09-16T02:04:29.638192+0000","last_peered":"2024-09-16T02:04:29.638192+0000","last_clean":"2024-09-16T02:04:29.638192+0000","last_became_active":"2024-09-16T02:03:34.737391+0000","last_became_peered":"2024-09-16T02:03:34.737391+0000","last_unstale":"2024-09-16T02:04:29.638192+0000","last_undegraded":"2024-09-16T02:04:29.638192+0000","last_fullsized":"2024-09-16T02:04:29.638192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:39.214180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718894+0000","last_change":"2024-09-16T02:03:37.049705+0000","last_active":"2024-09-16T02:04:29.718894+0000","last_peered":"2024-09-16T02:04:29.718894+0000","last_clean":"2024-09-16T02:04:29.718894+0000","last_became_active":"2024-09-16T02:03:37.049392+0000","last_became_peered":"2024-09-16T02:03:37.049392+0000","last_unstale":"2024-09-16T02:04:29.718894+0000","last_undegraded":"2024-09-16T02:04:29.718894+0000","last_fullsized":"2024-09-16T02:04:29.718894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.932391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":80,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:41.658339+0000","last_change":"2024-09-16T02:03:34.742268+0000","last_active":"2024-09-16T02:04:41.658339+0000","last_peered":"2024-09-16T02:04:41.658339+0000","last_clean":"2024-09-16T02:04:41.658339+0000","last_became_active":"2024-09-16T02:03:34.741934+0000","last_became_peered":"2024-09-16T02:03:34.741934+0000","last_unstale":"2024-09-16T02:04:41.658339+0000","last_undegraded":"2024-09-16T02:04:41.658339+0000","last_fullsized":"2024-09-16T02:04:41.658339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:30.301552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637511+0000","last_change":"2024-09-16T02:03:37.059119+0000","last_active":"2024-09-16T02:04:29.637511+0000","last_peered":"2024-09-16T02:04:29.637511+0000","last_clean":"2024-09-16T02:04:29.637511+0000","last_became_active":"2024-09-16T02:03:37.058738+0000","last_became_peered":"2024-09-16T02:03:37.058738+0000","last_unstale":"2024-09-16T02:04:29.637511+0000","last_undegraded":"2024-09-16T02:04:29.637511+0000","last_fullsized":"2024-09-16T02:04:29.637511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.496053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.642379+0000","last_change":"2024-09-16T02:04:29.642379+0000","last_active":"2024-09-16T02:04:29.642379+0000","last_peered":"2024-09-16T02:04:29.642379+0000","last_clean":"2024-09-16T02:04:29.642379+0000","last_became_active":"2024-09-16T02:04:29.638587+0000","last_became_peered":"2024-09-16T02:04:29.638587+0000","last_unstale":"2024-09-16T02:04:29.642379+0000","last_undegraded":"2024-09-16T02:04:29.642379+0000","last_fullsized":"2024-09-16T02:04:29.642379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:50.268223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631290+0000","last_change":"2024-09-16T02:03:37.050349+0000","last_active":"2024-09-16T02:04:29.631290+0000","last_peered":"2024-09-16T02:04:29.631290+0000","last_clean":"2024-09-16T02:04:29.631290+0000","last_became_active":"2024-09-16T02:03:37.049978+0000","last_became_peered":"2024-09-16T02:03:37.049978+0000","last_unstale":"2024-09-16T02:04:29.631290+0000","last_undegraded":"2024-09-16T02:04:29.631290+0000","last_fullsized":"2024-09-16T02:04:29.631290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:45.119683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637417+0000","last_change":"2024-09-16T02:03:34.740246+0000","last_active":"2024-09-16T02:04:29.637417+0000","last_peered":"2024-09-16T02:04:29.637417+0000","last_clean":"2024-09-16T02:04:29.637417+0000","last_became_active":"2024-09-16T02:03:34.739908+0000","last_became_peered":"2024-09-16T02:03:34.739908+0000","last_unstale":"2024-09-16T02:04:29.637417+0000","last_undegraded":"2024-09-16T02:04:29.637417+0000","last_fullsized":"2024-09-16T02:04:29.637417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:35.299733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636954+0000","last_change":"2024-09-16T02:03:37.056562+0000","last_active":"2024-09-16T02:04:29.636954+0000","last_peered":"2024-09-16T02:04:29.636954+0000","last_clean":"2024-09-16T02:04:29.636954+0000","last_became_active":"2024-09-16T02:03:37.056226+0000","last_became_peered":"2024-09-16T02:03:37.056226+0000","last_unstale":"2024-09-16T02:04:29.636954+0000","last_undegraded":"2024-09-16T02:04:29.636954+0000","last_fullsized":"2024-09-16T02:04:29.636954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.741538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'28","reported_seq":169,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:42.659738+0000","last_change":"2024-09-16T02:03:34.736403+0000","last_active":"2024-09-16T02:04:42.659738+0000","last_peered":"2024-09-16T02:04:42.659738+0000","last_clean":"2024-09-16T02:04:42.659738+0000","last_became_active":"2024-09-16T02:03:34.736074+0000","last_became_peered":"2024-09-16T02:03:34.736074+0000","last_unstale":"2024-09-16T02:04:42.659738+0000","last_undegraded":"2024-09-16T02:04:42.659738+0000","last_fullsized":"2024-09-16T02:04:42.659738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:34.769862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":119,"num_read_kb":117,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632824+0000","last_change":"2024-09-16T02:03:37.046803+0000","last_active":"2024-09-16T02:04:29.632824+0000","last_peered":"2024-09-16T02:04:29.632824+0000","last_clean":"2024-09-16T02:04:29.632824+0000","last_became_active":"2024-09-16T02:03:37.046459+0000","last_became_peered":"2024-09-16T02:03:37.046459+0000","last_unstale":"2024-09-16T02:04:29.632824+0000","last_undegraded":"2024-09-16T02:04:29.632824+0000","last_fullsized":"2024-09-16T02:04:29.632824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:55:38.460795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632119+0000","last_change":"2024-09-16T02:03:34.739016+0000","last_active":"2024-09-16T02:04:29.632119+0000","last_peered":"2024-09-16T02:04:29.632119+0000","last_clean":"2024-09-16T02:04:29.632119+0000","last_became_active":"2024-09-16T02:03:34.738583+0000","last_became_peered":"2024-09-16T02:03:34.738583+0000","last_unstale":"2024-09-16T02:04:29.632119+0000","last_undegraded":"2024-09-16T02:04:29.632119+0000","last_fullsized":"2024-09-16T02:04:29.632119+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:08.255832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628509+0000","last_change":"2024-09-16T02:03:37.053353+0000","last_active":"2024-09-16T02:04:29.628509+0000","last_peered":"2024-09-16T02:04:29.628509+0000","last_clean":"2024-09-16T02:04:29.628509+0000","last_became_active":"2024-09-16T02:03:37.051140+0000","last_became_peered":"2024-09-16T02:03:37.051140+0000","last_unstale":"2024-09-16T02:04:29.628509+0000","last_undegraded":"2024-09-16T02:04:29.628509+0000","last_fullsized":"2024-09-16T02:04:29.628509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:34.636338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636210+0000","last_change":"2024-09-16T02:03:34.742111+0000","last_active":"2024-09-16T02:04:29.636210+0000","last_peered":"2024-09-16T02:04:29.636210+0000","last_clean":"2024-09-16T02:04:29.636210+0000","last_became_active":"2024-09-16T02:03:34.741816+0000","last_became_peered":"2024-09-16T02:03:34.741816+0000","last_unstale":"2024-09-16T02:04:29.636210+0000","last_undegraded":"2024-09-16T02:04:29.636210+0000","last_fullsized":"2024-09-16T02:04:29.636210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:36.861995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632976+0000","last_change":"2024-09-16T02:03:37.047152+0000","last_active":"2024-09-16T02:04:29.632976+0000","last_peered":"2024-09-16T02:04:29.632976+0000","last_clean":"2024-09-16T02:04:29.632976+0000","last_became_active":"2024-09-16T02:03:37.046675+0000","last_became_peered":"2024-09-16T02:03:37.046675+0000","last_unstale":"2024-09-16T02:04:29.632976+0000","last_undegraded":"2024-09-16T02:04:29.632976+0000","last_fullsized":"2024-09-16T02:04:29.632976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:58.236519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":84,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:41.182933+0000","last_change":"2024-09-16T02:03:34.738487+0000","last_active":"2024-09-16T02:04:41.182933+0000","last_peered":"2024-09-16T02:04:41.182933+0000","last_clean":"2024-09-16T02:04:41.182933+0000","last_became_active":"2024-09-16T02:03:34.737061+0000","last_became_peered":"2024-09-16T02:03:34.737061+0000","last_unstale":"2024-09-16T02:04:41.182933+0000","last_undegraded":"2024-09-16T02:04:41.182933+0000","last_fullsized":"2024-09-16T02:04:41.182933+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:24.231640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632969+0000","last_change":"2024-09-16T02:03:37.044260+0000","last_active":"2024-09-16T02:04:29.632969+0000","last_peered":"2024-09-16T02:04:29.632969+0000","last_clean":"2024-09-16T02:04:29.632969+0000","last_became_active":"2024-09-16T02:03:37.043932+0000","last_became_peered":"2024-09-16T02:03:37.043932+0000","last_unstale":"2024-09-16T02:04:29.632969+0000","last_undegraded":"2024-09-16T02:04:29.632969+0000","last_fullsized":"2024-09-16T02:04:29.632969+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:55.718155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.651235+0000","last_change":"2024-09-16T02:04:29.651235+0000","last_active":"2024-09-16T02:04:29.651235+0000","last_peered":"2024-09-16T02:04:29.651235+0000","last_clean":"2024-09-16T02:04:29.651235+0000","last_became_active":"2024-09-16T02:04:29.633999+0000","last_became_peered":"2024-09-16T02:04:29.633999+0000","last_unstale":"2024-09-16T02:04:29.651235+0000","last_undegraded":"2024-09-16T02:04:29.651235+0000","last_fullsized":"2024-09-16T02:04:29.651235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.616990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636190+0000","last_change":"2024-09-16T02:03:37.053040+0000","last_active":"2024-09-16T02:04:29.636190+0000","last_peered":"2024-09-16T02:04:29.636190+0000","last_clean":"2024-09-16T02:04:29.636190+0000","last_became_active":"2024-09-16T02:03:37.052185+0000","last_became_peered":"2024-09-16T02:03:37.052185+0000","last_unstale":"2024-09-16T02:04:29.636190+0000","last_undegraded":"2024-09-16T02:04:29.636190+0000","last_fullsized":"2024-09-16T02:04:29.636190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:27.676678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.651376+0000","last_change":"2024-09-16T02:04:29.651376+0000","last_active":"2024-09-16T02:04:29.651376+0000","last_peered":"2024-09-16T02:04:29.651376+0000","last_clean":"2024-09-16T02:04:29.651376+0000","last_became_active":"2024-09-16T02:04:29.640260+0000","last_became_peered":"2024-09-16T02:04:29.640260+0000","last_unstale":"2024-09-16T02:04:29.651376+0000","last_undegraded":"2024-09-16T02:04:29.651376+0000","last_fullsized":"2024-09-16T02:04:29.651376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:39.426183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629483+0000","last_change":"2024-09-16T02:03:37.049420+0000","last_active":"2024-09-16T02:04:29.629483+0000","last_peered":"2024-09-16T02:04:29.629483+0000","last_clean":"2024-09-16T02:04:29.629483+0000","last_became_active":"2024-09-16T02:03:37.047671+0000","last_became_peered":"2024-09-16T02:03:37.047671+0000","last_unstale":"2024-09-16T02:04:29.629483+0000","last_undegraded":"2024-09-16T02:04:29.629483+0000","last_fullsized":"2024-09-16T02:04:29.629483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:23.023791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628718+0000","last_change":"2024-09-16T02:03:34.738516+0000","last_active":"2024-09-16T02:04:29.628718+0000","last_peered":"2024-09-16T02:04:29.628718+0000","last_clean":"2024-09-16T02:04:29.628718+0000","last_became_active":"2024-09-16T02:03:34.737701+0000","last_became_peered":"2024-09-16T02:03:34.737701+0000","last_unstale":"2024-09-16T02:04:29.628718+0000","last_undegraded":"2024-09-16T02:04:29.628718+0000","last_fullsized":"2024-09-16T02:04:29.628718+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:42.101865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633435+0000","last_change":"2024-09-16T02:03:37.046820+0000","last_active":"2024-09-16T02:04:29.633435+0000","last_peered":"2024-09-16T02:04:29.633435+0000","last_clean":"2024-09-16T02:04:29.633435+0000","last_became_active":"2024-09-16T02:03:37.046082+0000","last_became_peered":"2024-09-16T02:03:37.046082+0000","last_unstale":"2024-09-16T02:04:29.633435+0000","last_undegraded":"2024-09-16T02:04:29.633435+0000","last_fullsized":"2024-09-16T02:04:29.633435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:21.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637516+0000","last_change":"2024-09-16T02:03:34.741908+0000","last_active":"2024-09-16T02:04:29.637516+0000","last_peered":"2024-09-16T02:04:29.637516+0000","last_clean":"2024-09-16T02:04:29.637516+0000","last_became_active":"2024-09-16T02:03:34.739826+0000","last_became_peered":"2024-09-16T02:03:34.739826+0000","last_unstale":"2024-09-16T02:04:29.637516+0000","last_undegraded":"2024-09-16T02:04:29.637516+0000","last_fullsized":"2024-09-16T02:04:29.637516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:04.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633229+0000","last_change":"2024-09-16T02:03:37.056459+0000","last_active":"2024-09-16T02:04:29.633229+0000","last_peered":"2024-09-16T02:04:29.633229+0000","last_clean":"2024-09-16T02:04:29.633229+0000","last_became_active":"2024-09-16T02:03:37.055902+0000","last_became_peered":"2024-09-16T02:03:37.055902+0000","last_unstale":"2024-09-16T02:04:29.633229+0000","last_undegraded":"2024-09-16T02:04:29.633229+0000","last_fullsized":"2024-09-16T02:04:29.633229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:48:30.815654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632874+0000","last_change":"2024-09-16T02:03:34.741207+0000","last_active":"2024-09-16T02:04:29.632874+0000","last_peered":"2024-09-16T02:04:29.632874+0000","last_clean":"2024-09-16T02:04:29.632874+0000","last_became_active":"2024-09-16T02:03:34.740523+0000","last_became_peered":"2024-09-16T02:03:34.740523+0000","last_unstale":"2024-09-16T02:04:29.632874+0000","last_undegraded":"2024-09-16T02:04:29.632874+0000","last_fullsized":"2024-09-16T02:04:29.632874+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:36.199332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633942+0000","last_change":"2024-09-16T02:03:37.052792+0000","last_active":"2024-09-16T02:04:29.633942+0000","last_peered":"2024-09-16T02:04:29.633942+0000","last_clean":"2024-09-16T02:04:29.633942+0000","last_became_active":"2024-09-16T02:03:37.052301+0000","last_became_peered":"2024-09-16T02:03:37.052301+0000","last_unstale":"2024-09-16T02:04:29.633942+0000","last_undegraded":"2024-09-16T02:04:29.633942+0000","last_fullsized":"2024-09-16T02:04:29.633942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:55.197481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":333,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633949+0000","last_change":"2024-09-16T02:02:21.095893+0000","last_active":"2024-09-16T02:04:29.633949+0000","last_peered":"2024-09-16T02:04:29.633949+0000","last_clean":"2024-09-16T02:04:29.633949+0000","last_became_active":"2024-09-16T02:02:21.095395+0000","last_became_peered":"2024-09-16T02:02:21.095395+0000","last_unstale":"2024-09-16T02:04:29.633949+0000","last_undegraded":"2024-09-16T02:04:29.633949+0000","last_fullsized":"2024-09-16T02:04:29.633949+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_clean_scrub_stamp":"2024-09-16T01:58:50.954254+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:40.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638205+0000","last_change":"2024-09-16T02:03:34.746154+0000","last_active":"2024-09-16T02:04:29.638205+0000","last_peered":"2024-09-16T02:04:29.638205+0000","last_clean":"2024-09-16T02:04:29.638205+0000","last_became_active":"2024-09-16T02:03:34.745646+0000","last_became_peered":"2024-09-16T02:03:34.745646+0000","last_unstale":"2024-09-16T02:04:29.638205+0000","last_undegraded":"2024-09-16T02:04:29.638205+0000","last_fullsized":"2024-09-16T02:04:29.638205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:06.790000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718478+0000","last_change":"2024-09-16T02:03:37.056638+0000","last_active":"2024-09-16T02:04:29.718478+0000","last_peered":"2024-09-16T02:04:29.718478+0000","last_clean":"2024-09-16T02:04:29.718478+0000","last_became_active":"2024-09-16T02:03:37.056043+0000","last_became_peered":"2024-09-16T02:03:37.056043+0000","last_unstale":"2024-09-16T02:04:29.718478+0000","last_undegraded":"2024-09-16T02:04:29.718478+0000","last_fullsized":"2024-09-16T02:04:29.718478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:35:48.353380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.647682+0000","last_change":"2024-09-16T02:04:29.647682+0000","last_active":"2024-09-16T02:04:29.647682+0000","last_peered":"2024-09-16T02:04:29.647682+0000","last_clean":"2024-09-16T02:04:29.647682+0000","last_became_active":"2024-09-16T02:04:29.640065+0000","last_became_peered":"2024-09-16T02:04:29.640065+0000","last_unstale":"2024-09-16T02:04:29.647682+0000","last_undegraded":"2024-09-16T02:04:29.647682+0000","last_fullsized":"2024-09-16T02:04:29.647682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:53.765121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628419+0000","last_change":"2024-09-16T02:03:37.050895+0000","last_active":"2024-09-16T02:04:29.628419+0000","last_peered":"2024-09-16T02:04:29.628419+0000","last_clean":"2024-09-16T02:04:29.628419+0000","last_became_active":"2024-09-16T02:03:37.050559+0000","last_became_peered":"2024-09-16T02:03:37.050559+0000","last_unstale":"2024-09-16T02:04:29.628419+0000","last_undegraded":"2024-09-16T02:04:29.628419+0000","last_fullsized":"2024-09-16T02:04:29.628419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:20.067954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":84,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:44.186460+0000","last_change":"2024-09-16T02:03:34.740286+0000","last_active":"2024-09-16T02:04:44.186460+0000","last_peered":"2024-09-16T02:04:44.186460+0000","last_clean":"2024-09-16T02:04:44.186460+0000","last_became_active":"2024-09-16T02:03:34.739892+0000","last_became_peered":"2024-09-16T02:03:34.739892+0000","last_unstale":"2024-09-16T02:04:44.186460+0000","last_undegraded":"2024-09-16T02:04:44.186460+0000","last_fullsized":"2024-09-16T02:04:44.186460+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:59:14.809991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638092+0000","last_change":"2024-09-16T02:03:37.042207+0000","last_active":"2024-09-16T02:04:29.638092+0000","last_peered":"2024-09-16T02:04:29.638092+0000","last_clean":"2024-09-16T02:04:29.638092+0000","last_became_active":"2024-09-16T02:03:37.041283+0000","last_became_peered":"2024-09-16T02:03:37.041283+0000","last_unstale":"2024-09-16T02:04:29.638092+0000","last_undegraded":"2024-09-16T02:04:29.638092+0000","last_fullsized":"2024-09-16T02:04:29.638092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:45.569721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633269+0000","last_change":"2024-09-16T02:03:34.737834+0000","last_active":"2024-09-16T02:04:29.633269+0000","last_peered":"2024-09-16T02:04:29.633269+0000","last_clean":"2024-09-16T02:04:29.633269+0000","last_became_active":"2024-09-16T02:03:34.736706+0000","last_became_peered":"2024-09-16T02:03:34.736706+0000","last_unstale":"2024-09-16T02:04:29.633269+0000","last_undegraded":"2024-09-16T02:04:29.633269+0000","last_fullsized":"2024-09-16T02:04:29.633269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:27.037351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.630069+0000","last_change":"2024-09-16T02:03:37.060888+0000","last_active":"2024-09-16T02:04:29.630069+0000","last_peered":"2024-09-16T02:04:29.630069+0000","last_clean":"2024-09-16T02:04:29.630069+0000","last_became_active":"2024-09-16T02:03:37.060458+0000","last_became_peered":"2024-09-16T02:03:37.060458+0000","last_unstale":"2024-09-16T02:04:29.630069+0000","last_undegraded":"2024-09-16T02:04:29.630069+0000","last_fullsized":"2024-09-16T02:04:29.630069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:37.425376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.719071+0000","last_change":"2024-09-16T02:03:34.742298+0000","last_active":"2024-09-16T02:04:29.719071+0000","last_peered":"2024-09-16T02:04:29.719071+0000","last_clean":"2024-09-16T02:04:29.719071+0000","last_became_active":"2024-09-16T02:03:34.741960+0000","last_became_peered":"2024-09-16T02:03:34.741960+0000","last_unstale":"2024-09-16T02:04:29.719071+0000","last_undegraded":"2024-09-16T02:04:29.719071+0000","last_fullsized":"2024-09-16T02:04:29.719071+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:12.832207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629969+0000","last_change":"2024-09-16T02:03:37.060553+0000","last_active":"2024-09-16T02:04:29.629969+0000","last_peered":"2024-09-16T02:04:29.629969+0000","last_clean":"2024-09-16T02:04:29.629969+0000","last_became_active":"2024-09-16T02:03:37.059961+0000","last_became_peered":"2024-09-16T02:03:37.059961+0000","last_unstale":"2024-09-16T02:04:29.629969+0000","last_undegraded":"2024-09-16T02:04:29.629969+0000","last_fullsized":"2024-09-16T02:04:29.629969+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:22.904190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633669+0000","last_change":"2024-09-16T02:03:34.747828+0000","last_active":"2024-09-16T02:04:29.633669+0000","last_peered":"2024-09-16T02:04:29.633669+0000","last_clean":"2024-09-16T02:04:29.633669+0000","last_became_active":"2024-09-16T02:03:34.747548+0000","last_became_peered":"2024-09-16T02:03:34.747548+0000","last_unstale":"2024-09-16T02:04:29.633669+0000","last_undegraded":"2024-09-16T02:04:29.633669+0000","last_fullsized":"2024-09-16T02:04:29.633669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:49.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.628783+0000","last_change":"2024-09-16T02:03:37.051477+0000","last_active":"2024-09-16T02:04:29.628783+0000","last_peered":"2024-09-16T02:04:29.628783+0000","last_clean":"2024-09-16T02:04:29.628783+0000","last_became_active":"2024-09-16T02:03:37.048063+0000","last_became_peered":"2024-09-16T02:03:37.048063+0000","last_unstale":"2024-09-16T02:04:29.628783+0000","last_undegraded":"2024-09-16T02:04:29.628783+0000","last_fullsized":"2024-09-16T02:04:29.628783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:15.973972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.719147+0000","last_change":"2024-09-16T02:03:34.742935+0000","last_active":"2024-09-16T02:04:29.719147+0000","last_peered":"2024-09-16T02:04:29.719147+0000","last_clean":"2024-09-16T02:04:29.719147+0000","last_became_active":"2024-09-16T02:03:34.742611+0000","last_became_peered":"2024-09-16T02:03:34.742611+0000","last_unstale":"2024-09-16T02:04:29.719147+0000","last_undegraded":"2024-09-16T02:04:29.719147+0000","last_fullsized":"2024-09-16T02:04:29.719147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:36.695220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718255+0000","last_change":"2024-09-16T02:03:37.054926+0000","last_active":"2024-09-16T02:04:29.718255+0000","last_peered":"2024-09-16T02:04:29.718255+0000","last_clean":"2024-09-16T02:04:29.718255+0000","last_became_active":"2024-09-16T02:03:37.054483+0000","last_became_peered":"2024-09-16T02:03:37.054483+0000","last_unstale":"2024-09-16T02:04:29.718255+0000","last_undegraded":"2024-09-16T02:04:29.718255+0000","last_fullsized":"2024-09-16T02:04:29.718255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.638845+0000","last_change":"2024-09-16T02:03:34.739611+0000","last_active":"2024-09-16T02:04:29.638845+0000","last_peered":"2024-09-16T02:04:29.638845+0000","last_clean":"2024-09-16T02:04:29.638845+0000","last_became_active":"2024-09-16T02:03:34.738090+0000","last_became_peered":"2024-09-16T02:03:34.738090+0000","last_unstale":"2024-09-16T02:04:29.638845+0000","last_undegraded":"2024-09-16T02:04:29.638845+0000","last_fullsized":"2024-09-16T02:04:29.638845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:47.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631158+0000","last_change":"2024-09-16T02:03:37.053020+0000","last_active":"2024-09-16T02:04:29.631158+0000","last_peered":"2024-09-16T02:04:29.631158+0000","last_clean":"2024-09-16T02:04:29.631158+0000","last_became_active":"2024-09-16T02:03:37.052365+0000","last_became_peered":"2024-09-16T02:03:37.052365+0000","last_unstale":"2024-09-16T02:04:29.631158+0000","last_undegraded":"2024-09-16T02:04:29.631158+0000","last_fullsized":"2024-09-16T02:04:29.631158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:54.965028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636913+0000","last_change":"2024-09-16T02:03:34.740726+0000","last_active":"2024-09-16T02:04:29.636913+0000","last_peered":"2024-09-16T02:04:29.636913+0000","last_clean":"2024-09-16T02:04:29.636913+0000","last_became_active":"2024-09-16T02:03:34.740402+0000","last_became_peered":"2024-09-16T02:03:34.740402+0000","last_unstale":"2024-09-16T02:04:29.636913+0000","last_undegraded":"2024-09-16T02:04:29.636913+0000","last_fullsized":"2024-09-16T02:04:29.636913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:05.761646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631417+0000","last_change":"2024-09-16T02:03:37.044862+0000","last_active":"2024-09-16T02:04:29.631417+0000","last_peered":"2024-09-16T02:04:29.631417+0000","last_clean":"2024-09-16T02:04:29.631417+0000","last_became_active":"2024-09-16T02:03:37.044406+0000","last_became_peered":"2024-09-16T02:03:37.044406+0000","last_unstale":"2024-09-16T02:04:29.631417+0000","last_undegraded":"2024-09-16T02:04:29.631417+0000","last_fullsized":"2024-09-16T02:04:29.631417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:03.326380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718201+0000","last_change":"2024-09-16T02:03:34.745505+0000","last_active":"2024-09-16T02:04:29.718201+0000","last_peered":"2024-09-16T02:04:29.718201+0000","last_clean":"2024-09-16T02:04:29.718201+0000","last_became_active":"2024-09-16T02:03:34.745151+0000","last_became_peered":"2024-09-16T02:03:34.745151+0000","last_unstale":"2024-09-16T02:04:29.718201+0000","last_undegraded":"2024-09-16T02:04:29.718201+0000","last_fullsized":"2024-09-16T02:04:29.718201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:57.401176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.629299+0000","last_change":"2024-09-16T02:03:37.046692+0000","last_active":"2024-09-16T02:04:29.629299+0000","last_peered":"2024-09-16T02:04:29.629299+0000","last_clean":"2024-09-16T02:04:29.629299+0000","last_became_active":"2024-09-16T02:03:37.046302+0000","last_became_peered":"2024-09-16T02:03:37.046302+0000","last_unstale":"2024-09-16T02:04:29.629299+0000","last_undegraded":"2024-09-16T02:04:29.629299+0000","last_fullsized":"2024-09-16T02:04:29.629299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:47.727431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632142+0000","last_change":"2024-09-16T02:03:34.739847+0000","last_active":"2024-09-16T02:04:29.632142+0000","last_peered":"2024-09-16T02:04:29.632142+0000","last_clean":"2024-09-16T02:04:29.632142+0000","last_became_active":"2024-09-16T02:03:34.739514+0000","last_became_peered":"2024-09-16T02:03:34.739514+0000","last_unstale":"2024-09-16T02:04:29.632142+0000","last_undegraded":"2024-09-16T02:04:29.632142+0000","last_fullsized":"2024-09-16T02:04:29.632142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:44.617127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718141+0000","last_change":"2024-09-16T02:03:37.054910+0000","last_active":"2024-09-16T02:04:29.718141+0000","last_peered":"2024-09-16T02:04:29.718141+0000","last_clean":"2024-09-16T02:04:29.718141+0000","last_became_active":"2024-09-16T02:03:37.053852+0000","last_became_peered":"2024-09-16T02:03:37.053852+0000","last_unstale":"2024-09-16T02:04:29.718141+0000","last_undegraded":"2024-09-16T02:04:29.718141+0000","last_fullsized":"2024-09-16T02:04:29.718141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:29.042154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.632406+0000","last_change":"2024-09-16T02:03:34.743027+0000","last_active":"2024-09-16T02:04:29.632406+0000","last_peered":"2024-09-16T02:04:29.632406+0000","last_clean":"2024-09-16T02:04:29.632406+0000","last_became_active":"2024-09-16T02:03:34.742658+0000","last_became_peered":"2024-09-16T02:03:34.742658+0000","last_unstale":"2024-09-16T02:04:29.632406+0000","last_undegraded":"2024-09-16T02:04:29.632406+0000","last_fullsized":"2024-09-16T02:04:29.632406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:36.779568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.633061+0000","last_change":"2024-09-16T02:03:37.052783+0000","last_active":"2024-09-16T02:04:29.633061+0000","last_peered":"2024-09-16T02:04:29.633061+0000","last_clean":"2024-09-16T02:04:29.633061+0000","last_became_active":"2024-09-16T02:03:37.052409+0000","last_became_peered":"2024-09-16T02:03:37.052409+0000","last_unstale":"2024-09-16T02:04:29.633061+0000","last_undegraded":"2024-09-16T02:04:29.633061+0000","last_fullsized":"2024-09-16T02:04:29.633061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:52.667379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.637314+0000","last_change":"2024-09-16T02:03:37.053645+0000","last_active":"2024-09-16T02:04:29.637314+0000","last_peered":"2024-09-16T02:04:29.637314+0000","last_clean":"2024-09-16T02:04:29.637314+0000","last_became_active":"2024-09-16T02:03:37.053340+0000","last_became_peered":"2024-09-16T02:03:37.053340+0000","last_unstale":"2024-09-16T02:04:29.637314+0000","last_undegraded":"2024-09-16T02:04:29.637314+0000","last_fullsized":"2024-09-16T02:04:29.637314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:59.827249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631820+0000","last_change":"2024-09-16T02:03:34.744240+0000","last_active":"2024-09-16T02:04:29.631820+0000","last_peered":"2024-09-16T02:04:29.631820+0000","last_clean":"2024-09-16T02:04:29.631820+0000","last_became_active":"2024-09-16T02:03:34.743898+0000","last_became_peered":"2024-09-16T02:03:34.743898+0000","last_unstale":"2024-09-16T02:04:29.631820+0000","last_undegraded":"2024-09-16T02:04:29.631820+0000","last_fullsized":"2024-09-16T02:04:29.631820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:37.820529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.636830+0000","last_change":"2024-09-16T02:03:37.055769+0000","last_active":"2024-09-16T02:04:29.636830+0000","last_peered":"2024-09-16T02:04:29.636830+0000","last_clean":"2024-09-16T02:04:29.636830+0000","last_became_active":"2024-09-16T02:03:37.054458+0000","last_became_peered":"2024-09-16T02:03:37.054458+0000","last_unstale":"2024-09-16T02:04:29.636830+0000","last_undegraded":"2024-09-16T02:04:29.636830+0000","last_fullsized":"2024-09-16T02:04:29.636830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:30.218712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.647534+0000","last_change":"2024-09-16T02:04:29.647534+0000","last_active":"2024-09-16T02:04:29.647534+0000","last_peered":"2024-09-16T02:04:29.647534+0000","last_clean":"2024-09-16T02:04:29.647534+0000","last_became_active":"2024-09-16T02:04:29.640317+0000","last_became_peered":"2024-09-16T02:04:29.640317+0000","last_unstale":"2024-09-16T02:04:29.647534+0000","last_undegraded":"2024-09-16T02:04:29.647534+0000","last_fullsized":"2024-09-16T02:04:29.647534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:14:33.490508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,6],"acting":[8,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.631287+0000","last_change":"2024-09-16T02:03:37.054370+0000","last_active":"2024-09-16T02:04:29.631287+0000","last_peered":"2024-09-16T02:04:29.631287+0000","last_clean":"2024-09-16T02:04:29.631287+0000","last_became_active":"2024-09-16T02:03:37.054051+0000","last_became_peered":"2024-09-16T02:03:37.054051+0000","last_unstale":"2024-09-16T02:04:29.631287+0000","last_undegraded":"2024-09-16T02:04:29.631287+0000","last_fullsized":"2024-09-16T02:04:29.631287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:21:42.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718654+0000","last_change":"2024-09-16T02:03:34.739696+0000","last_active":"2024-09-16T02:04:29.718654+0000","last_peered":"2024-09-16T02:04:29.718654+0000","last_clean":"2024-09-16T02:04:29.718654+0000","last_became_active":"2024-09-16T02:03:34.738867+0000","last_became_peered":"2024-09-16T02:03:34.738867+0000","last_unstale":"2024-09-16T02:04:29.718654+0000","last_undegraded":"2024-09-16T02:04:29.718654+0000","last_fullsized":"2024-09-16T02:04:29.718654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:30.753970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.718201+0000","last_change":"2024-09-16T02:03:37.058118+0000","last_active":"2024-09-16T02:04:29.718201+0000","last_peered":"2024-09-16T02:04:29.718201+0000","last_clean":"2024-09-16T02:04:29.718201+0000","last_became_active":"2024-09-16T02:03:37.057391+0000","last_became_peered":"2024-09-16T02:03:37.057391+0000","last_unstale":"2024-09-16T02:04:29.718201+0000","last_undegraded":"2024-09-16T02:04:29.718201+0000","last_fullsized":"2024-09-16T02:04:29.718201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:39.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T02:04:29.719186+0000","last_change":"2024-09-16T02:03:34.742184+0000","last_active":"2024-09-16T02:04:29.719186+0000","last_peered":"2024-09-16T02:04:29.719186+0000","last_clean":"2024-09-16T02:04:29.719186+0000","last_became_active":"2024-09-16T02:03:34.741838+0000","last_became_peered":"2024-09-16T02:03:34.741838+0000","last_unstale":"2024-09-16T02:04:29.719186+0000","last_undegraded":"2024-09-16T02:04:29.719186+0000","last_fullsized":"2024-09-16T02:04:29.719186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:57.338380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":354,"num_read_kb":249,"num_write":164,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1149,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678044,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":423171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1564672,"data_stored":1074697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037797,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":884736,"data_stored":423295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266732,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":876544,"data_stored":423103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495664,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32636,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712516,"statfs":{"total":95995035648,"available":95961616384,"internally_reserved":0,"allocated":1478656,"data_stored":1015540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659189,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":422684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888121,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1486848,"data_stored":1014809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051647,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":884736,"data_stored":424529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280581,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":888832,"data_stored":422995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476810,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":950272,"data_stored":482969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607632,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":937984,"data_stored":482263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":892928,"data_stored":423955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":59786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:04:48.042 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T02:04:48.043 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T02:04:48.043 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T02:04:48.043 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T02:04:48.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:48 smithi103 ceph-mon[30398]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:48.416 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:48.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:48 smithi136 ceph-mon[35323]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:48.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:48 smithi172 ceph-mon[35340]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:04:48.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.942+0000 7fcb86c00640 1 -- 172.21.15.103:0/1145648817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb80074ee0 msgr2=0x7fcb8010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.942+0000 7fcb86c00640 1 --2- 172.21.15.103:0/1145648817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb80074ee0 0x7fcb8010def0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009900 tx=0x7fcb6802a480 comp rx=0 tx=0).stop 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.942+0000 7fcb86c00640 1 -- 172.21.15.103:0/1145648817 shutdown_connections 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.942+0000 7fcb86c00640 1 --2- 172.21.15.103:0/1145648817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb8010e430 0x7fcb80110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.942+0000 7fcb86c00640 1 --2- 172.21.15.103:0/1145648817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb80074ee0 0x7fcb8010def0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.942+0000 7fcb86c00640 1 --2- 172.21.15.103:0/1145648817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb800745c0 0x7fcb800749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.942+0000 7fcb86c00640 1 -- 172.21.15.103:0/1145648817 >> 172.21.15.103:0/1145648817 conn(0x7fcb800fd8b0 msgr2=0x7fcb800ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.943+0000 7fcb86c00640 1 -- 172.21.15.103:0/1145648817 shutdown_connections 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.943+0000 7fcb86c00640 1 -- 172.21.15.103:0/1145648817 wait complete. 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.944+0000 7fcb86c00640 1 Processor -- start 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.944+0000 7fcb86c00640 1 -- start start 2024-09-16T02:04:48.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.944+0000 7fcb86c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb800745c0 0x7fcb8019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.944+0000 7fcb86c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb80074ee0 0x7fcb8019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.944+0000 7fcb86c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb8010e430 0x7fcb8019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.944+0000 7fcb86c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb80114240 con 0x7fcb800745c0 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.944+0000 7fcb86c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801140c0 con 0x7fcb8010e430 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.944+0000 7fcb86c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801143c0 con 0x7fcb80074ee0 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb7fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb800745c0 0x7fcb8019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb84c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb8010e430 0x7fcb8019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb7fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb800745c0 0x7fcb8019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55806/0 (socket says 172.21.15.103:55806) 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb7fe00640 1 -- 172.21.15.103:0/1580630304 learned_addr learned my addr 172.21.15.103:0/1580630304 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb77400640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb80074ee0 0x7fcb8019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb7fe00640 1 -- 172.21.15.103:0/1580630304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb80074ee0 msgr2=0x7fcb8019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb7fe00640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb80074ee0 0x7fcb8019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:48.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb7fe00640 1 -- 172.21.15.103:0/1580630304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb8010e430 msgr2=0x7fcb8019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb7fe00640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb8010e430 0x7fcb8019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb7fe00640 1 -- 172.21.15.103:0/1580630304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8019f630 con 0x7fcb800745c0 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.945+0000 7fcb7fe00640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb800745c0 0x7fcb8019a680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000b4a0 tx=0x7fcb7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.946+0000 7fcb7d600640 1 -- 172.21.15.103:0/1580630304 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70014070 con 0x7fcb800745c0 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.946+0000 7fcb7d600640 1 -- 172.21.15.103:0/1580630304 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb70010af0 con 0x7fcb800745c0 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.946+0000 7fcb86c00640 1 -- 172.21.15.103:0/1580630304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8019f920 con 0x7fcb800745c0 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.946+0000 7fcb7d600640 1 -- 172.21.15.103:0/1580630304 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70017d10 con 0x7fcb800745c0 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.946+0000 7fcb86c00640 1 -- 172.21.15.103:0/1580630304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb801a7200 con 0x7fcb800745c0 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.947+0000 7fcb86c00640 1 -- 172.21.15.103:0/1580630304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb800749a0 con 0x7fcb800745c0 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.949+0000 7fcb7d600640 1 -- 172.21.15.103:0/1580630304 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcb70017070 con 0x7fcb800745c0 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.950+0000 7fcb7d600640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcb640a3350 0x7fcb640a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.950+0000 7fcb7d600640 1 -- 172.21.15.103:0/1580630304 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fcb700cecb0 con 0x7fcb800745c0 2024-09-16T02:04:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.950+0000 7fcb77400640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcb640a3350 0x7fcb640a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:48.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.951+0000 7fcb77400640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcb640a3350 0x7fcb640a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb68000c00 tx=0x7fcb68005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:48.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:48.954+0000 7fcb7d600640 1 -- 172.21.15.103:0/1580630304 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb7009b2c0 con 0x7fcb800745c0 2024-09-16T02:04:49.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.168+0000 7fcb86c00640 1 -- 172.21.15.103:0/1580630304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcb801a7400 con 0x7fcb800745c0 2024-09-16T02:04:49.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.169+0000 7fcb7d600640 1 -- 172.21.15.103:0/1580630304 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fcb7009f5c0 con 0x7fcb800745c0 2024-09-16T02:04:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:49.170 INFO:teuthology.orchestra.run.smithi103.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T02:04:49.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.171+0000 7fcb86c00640 1 -- 172.21.15.103:0/1580630304 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcb640a3350 msgr2=0x7fcb640a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:49.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.171+0000 7fcb86c00640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcb640a3350 0x7fcb640a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb68000c00 tx=0x7fcb68005df0 comp rx=0 tx=0).stop 2024-09-16T02:04:49.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.171+0000 7fcb86c00640 1 -- 172.21.15.103:0/1580630304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb800745c0 msgr2=0x7fcb8019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:49.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.171+0000 7fcb86c00640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb800745c0 0x7fcb8019a680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000b4a0 tx=0x7fcb7000b960 comp rx=0 tx=0).stop 2024-09-16T02:04:49.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.172+0000 7fcb86c00640 1 -- 172.21.15.103:0/1580630304 shutdown_connections 2024-09-16T02:04:49.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.172+0000 7fcb86c00640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcb640a3350 0x7fcb640a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:49.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.172+0000 7fcb86c00640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb8010e430 0x7fcb8019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:49.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.172+0000 7fcb86c00640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb80074ee0 0x7fcb8019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:49.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.172+0000 7fcb86c00640 1 --2- 172.21.15.103:0/1580630304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb800745c0 0x7fcb8019a680 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:49.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.172+0000 7fcb86c00640 1 -- 172.21.15.103:0/1580630304 >> 172.21.15.103:0/1580630304 conn(0x7fcb800fd8b0 msgr2=0x7fcb800fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:49.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.172+0000 7fcb86c00640 1 -- 172.21.15.103:0/1580630304 shutdown_connections 2024-09-16T02:04:49.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:49.172+0000 7fcb86c00640 1 -- 172.21.15.103:0/1580630304 wait complete. 2024-09-16T02:04:49.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:49 smithi103 ceph-mon[30398]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:04:49.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:49 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1580630304' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T02:04:49.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:49 smithi136 ceph-mon[35323]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:04:49.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:49 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1580630304' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T02:04:49.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:49 smithi172 ceph-mon[35340]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:04:49.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:49 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1580630304' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T02:04:49.892 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T02:04:49.893 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:04:50.262 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.759+0000 7f4d2e600640 1 -- 172.21.15.103:0/1689287851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28101de0 msgr2=0x7f4d281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.759+0000 7f4d2e600640 1 --2- 172.21.15.103:0/1689287851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28101de0 0x7f4d281021c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009900 tx=0x7f4d1802a410 comp rx=0 tx=0).stop 2024-09-16T02:04:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.759+0000 7f4d2e600640 1 -- 172.21.15.103:0/1689287851 shutdown_connections 2024-09-16T02:04:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.759+0000 7f4d2e600640 1 --2- 172.21.15.103:0/1689287851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d28106f10 0x7f4d2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.759+0000 7f4d2e600640 1 --2- 172.21.15.103:0/1689287851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d28102790 0x7f4d281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.759+0000 7f4d2e600640 1 --2- 172.21.15.103:0/1689287851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28101de0 0x7f4d281021c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.759+0000 7f4d2e600640 1 -- 172.21.15.103:0/1689287851 >> 172.21.15.103:0/1689287851 conn(0x7f4d280fd8c0 msgr2=0x7f4d280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.759+0000 7f4d2e600640 1 -- 172.21.15.103:0/1689287851 shutdown_connections 2024-09-16T02:04:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.759+0000 7f4d2e600640 1 -- 172.21.15.103:0/1689287851 wait complete. 2024-09-16T02:04:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.760+0000 7f4d2e600640 1 Processor -- start 2024-09-16T02:04:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.760+0000 7f4d2e600640 1 -- start start 2024-09-16T02:04:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.760+0000 7f4d2e600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28101de0 0x7f4d280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.760+0000 7f4d2e600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d28102790 0x7f4d28074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.760+0000 7f4d2e600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d28106f10 0x7f4d28074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.760+0000 7f4d2e600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28111f70 con 0x7f4d28101de0 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.760+0000 7f4d2e600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28111df0 con 0x7f4d28106f10 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.760+0000 7f4d2e600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281120f0 con 0x7f4d28102790 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28101de0 0x7f4d280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28101de0 0x7f4d280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55826/0 (socket says 172.21.15.103:55826) 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27400640 1 -- 172.21.15.103:0/3437971965 learned_addr learned my addr 172.21.15.103:0/3437971965 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d1ea00640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d28102790 0x7f4d28074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27400640 1 -- 172.21.15.103:0/3437971965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d28102790 msgr2=0x7f4d28074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27400640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d28102790 0x7f4d28074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27400640 1 -- 172.21.15.103:0/3437971965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d28106f10 msgr2=0x7f4d28074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27e00640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d28106f10 0x7f4d28074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27400640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d28106f10 0x7f4d28074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27400640 1 -- 172.21.15.103:0/3437971965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d281a6950 con 0x7f4d28101de0 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27e00640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d28106f10 0x7f4d28074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.761+0000 7f4d27400640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28101de0 0x7f4d280744f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4d18006010 tx=0x7f4d18004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.762+0000 7f4d24c00640 1 -- 172.21.15.103:0/3437971965 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1803f030 con 0x7f4d28101de0 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.762+0000 7f4d24c00640 1 -- 172.21.15.103:0/3437971965 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d18004440 con 0x7f4d28101de0 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.762+0000 7f4d2e600640 1 -- 172.21.15.103:0/3437971965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d281a6be0 con 0x7f4d28101de0 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.762+0000 7f4d24c00640 1 -- 172.21.15.103:0/3437971965 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18035070 con 0x7f4d28101de0 2024-09-16T02:04:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.762+0000 7f4d2e600640 1 -- 172.21.15.103:0/3437971965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d281a70c0 con 0x7f4d28101de0 2024-09-16T02:04:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.763+0000 7f4d2e600640 1 -- 172.21.15.103:0/3437971965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d281021c0 con 0x7f4d28101de0 2024-09-16T02:04:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.765+0000 7f4d24c00640 1 -- 172.21.15.103:0/3437971965 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4d18042030 con 0x7f4d28101de0 2024-09-16T02:04:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.766+0000 7f4d24c00640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d0c0a32d0 0x7f4d0c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.766+0000 7f4d24c00640 1 -- 172.21.15.103:0/3437971965 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f4d180efc60 con 0x7f4d28101de0 2024-09-16T02:04:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.766+0000 7f4d1ea00640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d0c0a32d0 0x7f4d0c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.767+0000 7f4d1ea00640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d0c0a32d0 0x7f4d0c0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d28075f80 tx=0x7f4d10005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.770+0000 7f4d24c00640 1 -- 172.21.15.103:0/3437971965 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d180bc7b0 con 0x7f4d28101de0 2024-09-16T02:04:50.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.982+0000 7f4d2e600640 1 -- 172.21.15.103:0/3437971965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d28075a20 con 0x7f4d28101de0 2024-09-16T02:04:50.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.983+0000 7f4d24c00640 1 -- 172.21.15.103:0/3437971965 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7f4d180bc0b0 con 0x7f4d28101de0 2024-09-16T02:04:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:50.984 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":23,"btime":"2024-09-16T02:03:50:098205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:03:49.812580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24505,24506,24545]},"id":1}]} 2024-09-16T02:04:50.985 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 23 2024-09-16T02:04:50.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 -- 172.21.15.103:0/3437971965 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d0c0a32d0 msgr2=0x7f4d0c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:50.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d0c0a32d0 0x7f4d0c0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d28075f80 tx=0x7f4d10005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:50.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 -- 172.21.15.103:0/3437971965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28101de0 msgr2=0x7f4d280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:50.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28101de0 0x7f4d280744f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4d18006010 tx=0x7f4d18004040 comp rx=0 tx=0).stop 2024-09-16T02:04:50.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 -- 172.21.15.103:0/3437971965 shutdown_connections 2024-09-16T02:04:50.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4d0c0a32d0 0x7f4d0c0a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:50.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d28106f10 0x7f4d28074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:50.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d28102790 0x7f4d28074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:50.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 --2- 172.21.15.103:0/3437971965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28101de0 0x7f4d280744f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:50.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 -- 172.21.15.103:0/3437971965 >> 172.21.15.103:0/3437971965 conn(0x7f4d280fd8c0 msgr2=0x7f4d280febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:50.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 -- 172.21.15.103:0/3437971965 shutdown_connections 2024-09-16T02:04:50.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:50.986+0000 7f4d2e600640 1 -- 172.21.15.103:0/3437971965 wait complete. 2024-09-16T02:04:51.570 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:51 smithi103 ceph-mon[30398]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:04:51.570 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:51 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3437971965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:51.610 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:04:51.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:51 smithi136 ceph-mon[35323]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:04:51.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:51 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3437971965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:51.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:51 smithi172 ceph-mon[35340]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:04:51.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:51 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3437971965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:51.982 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:52.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.509+0000 7ff91e000640 1 -- 172.21.15.103:0/2211580013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9181019d0 msgr2=0x7ff918101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:52.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.509+0000 7ff91e000640 1 --2- 172.21.15.103:0/2211580013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9181019d0 0x7ff918101e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff904009930 tx=0x7ff90402a420 comp rx=0 tx=0).stop 2024-09-16T02:04:52.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.509+0000 7ff91e000640 1 -- 172.21.15.103:0/2211580013 shutdown_connections 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.509+0000 7ff91e000640 1 --2- 172.21.15.103:0/2211580013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff918102370 0x7ff91810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.509+0000 7ff91e000640 1 --2- 172.21.15.103:0/2211580013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9181019d0 0x7ff918101e30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.509+0000 7ff91e000640 1 --2- 172.21.15.103:0/2211580013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9181079d0 0x7ff918107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.509+0000 7ff91e000640 1 -- 172.21.15.103:0/2211580013 >> 172.21.15.103:0/2211580013 conn(0x7ff9180fd710 msgr2=0x7ff9180ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.509+0000 7ff91e000640 1 -- 172.21.15.103:0/2211580013 shutdown_connections 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.509+0000 7ff91e000640 1 -- 172.21.15.103:0/2211580013 wait complete. 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.510+0000 7ff91e000640 1 Processor -- start 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.510+0000 7ff91e000640 1 -- start start 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.510+0000 7ff91e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9181019d0 0x7ff91819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.510+0000 7ff91e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff918102370 0x7ff91819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:52.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.510+0000 7ff91e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9181079d0 0x7ff91819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.510+0000 7ff91e000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff918111e60 con 0x7ff9181079d0 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.510+0000 7ff91e000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff918111ce0 con 0x7ff9181019d0 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.510+0000 7ff91e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff918111fe0 con 0x7ff918102370 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff91d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9181079d0 0x7ff91819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff91d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9181079d0 0x7ff91819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55852/0 (socket says 172.21.15.103:55852) 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff91d600640 1 -- 172.21.15.103:0/1561022086 learned_addr learned my addr 172.21.15.103:0/1561022086 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff917e00640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff918102370 0x7ff91819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff91cc00640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9181019d0 0x7ff91819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff91d600640 1 -- 172.21.15.103:0/1561022086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff918102370 msgr2=0x7ff91819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff91d600640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff918102370 0x7ff91819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff91d600640 1 -- 172.21.15.103:0/1561022086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9181019d0 msgr2=0x7ff91819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff91d600640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9181019d0 0x7ff91819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff91d600640 1 -- 172.21.15.103:0/1561022086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91819f4f0 con 0x7ff9181079d0 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.511+0000 7ff91d600640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9181079d0 0x7ff91819ee10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff91000eb10 tx=0x7ff91000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.512+0000 7ff915600640 1 -- 172.21.15.103:0/1561022086 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910004b20 con 0x7ff9181079d0 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.512+0000 7ff915600640 1 -- 172.21.15.103:0/1561022086 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff910004cc0 con 0x7ff9181079d0 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.512+0000 7ff915600640 1 -- 172.21.15.103:0/1561022086 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910010760 con 0x7ff9181079d0 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.512+0000 7ff91e000640 1 -- 172.21.15.103:0/1561022086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91819f7e0 con 0x7ff9181079d0 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.512+0000 7ff91e000640 1 -- 172.21.15.103:0/1561022086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9181a70c0 con 0x7ff9181079d0 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.513+0000 7ff91e000640 1 -- 172.21.15.103:0/1561022086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff918101e30 con 0x7ff9181079d0 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.515+0000 7ff915600640 1 -- 172.21.15.103:0/1561022086 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff910005020 con 0x7ff9181079d0 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.516+0000 7ff915600640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9000a3350 0x7ff9000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.516+0000 7ff915600640 1 -- 172.21.15.103:0/1561022086 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7ff9100d02c0 con 0x7ff9181079d0 2024-09-16T02:04:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.516+0000 7ff91cc00640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9000a3350 0x7ff9000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.517+0000 7ff91cc00640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9000a3350 0x7ff9000a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009890 tx=0x7ff90c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.520+0000 7ff915600640 1 -- 172.21.15.103:0/1561022086 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff91009ce10 con 0x7ff9181079d0 2024-09-16T02:04:52.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.732+0000 7ff91e000640 1 -- 172.21.15.103:0/1561022086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9181a73c0 con 0x7ff9181079d0 2024-09-16T02:04:52.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.733+0000 7ff915600640 1 -- 172.21.15.103:0/1561022086 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7ff91009c710 con 0x7ff9181079d0 2024-09-16T02:04:52.734 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:52.734 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":23,"btime":"2024-09-16T02:03:50:098205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:03:49.812580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24505,24506,24545]},"id":1}]} 2024-09-16T02:04:52.734 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 23 2024-09-16T02:04:52.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.735+0000 7ff91e000640 1 -- 172.21.15.103:0/1561022086 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9000a3350 msgr2=0x7ff9000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:52.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.735+0000 7ff91e000640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9000a3350 0x7ff9000a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009890 tx=0x7ff90c008040 comp rx=0 tx=0).stop 2024-09-16T02:04:52.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.735+0000 7ff91e000640 1 -- 172.21.15.103:0/1561022086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9181079d0 msgr2=0x7ff91819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:52.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.735+0000 7ff91e000640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9181079d0 0x7ff91819ee10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff91000eb10 tx=0x7ff91000c2d0 comp rx=0 tx=0).stop 2024-09-16T02:04:52.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.735+0000 7ff91e000640 1 -- 172.21.15.103:0/1561022086 shutdown_connections 2024-09-16T02:04:52.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.735+0000 7ff91e000640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9000a3350 0x7ff9000a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:52.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.735+0000 7ff91e000640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9181079d0 0x7ff91819ee10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:52.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.735+0000 7ff91e000640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff918102370 0x7ff91819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:52.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.735+0000 7ff91e000640 1 --2- 172.21.15.103:0/1561022086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9181019d0 0x7ff91819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:52.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.735+0000 7ff91e000640 1 -- 172.21.15.103:0/1561022086 >> 172.21.15.103:0/1561022086 conn(0x7ff9180fd710 msgr2=0x7ff9180fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:52.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.736+0000 7ff91e000640 1 -- 172.21.15.103:0/1561022086 shutdown_connections 2024-09-16T02:04:52.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:52.736+0000 7ff91e000640 1 -- 172.21.15.103:0/1561022086 wait complete. 2024-09-16T02:04:53.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:04:53.741 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:53.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:53 smithi103 ceph-mon[30398]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:04:53.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:53 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1561022086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:53 smithi172 ceph-mon[35340]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:04:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:53 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1561022086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:54.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:53 smithi136 ceph-mon[35323]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:04:54.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:53 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1561022086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.282+0000 7f39f2000640 1 -- 172.21.15.103:0/2394011709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39ec102790 msgr2=0x7f39ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.282+0000 7f39f2000640 1 --2- 172.21.15.103:0/2394011709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39ec102790 0x7f39ec1067e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009900 tx=0x7f39d402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.282+0000 7f39f2000640 1 -- 172.21.15.103:0/2394011709 shutdown_connections 2024-09-16T02:04:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.282+0000 7f39f2000640 1 --2- 172.21.15.103:0/2394011709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39ec106f10 0x7f39ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.282+0000 7f39f2000640 1 --2- 172.21.15.103:0/2394011709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39ec102790 0x7f39ec1067e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.282+0000 7f39f2000640 1 --2- 172.21.15.103:0/2394011709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39ec101de0 0x7f39ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.282+0000 7f39f2000640 1 -- 172.21.15.103:0/2394011709 >> 172.21.15.103:0/2394011709 conn(0x7f39ec0fd8c0 msgr2=0x7f39ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.282+0000 7f39f2000640 1 -- 172.21.15.103:0/2394011709 shutdown_connections 2024-09-16T02:04:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.282+0000 7f39f2000640 1 -- 172.21.15.103:0/2394011709 wait complete. 2024-09-16T02:04:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39f2000640 1 Processor -- start 2024-09-16T02:04:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39f2000640 1 -- start start 2024-09-16T02:04:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39f2000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39ec101de0 0x7f39ec19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39f2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39ec102790 0x7f39ec19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39f2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39ec106f10 0x7f39ec19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39f2000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec112000 con 0x7f39ec102790 2024-09-16T02:04:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39f2000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec111e80 con 0x7f39ec106f10 2024-09-16T02:04:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39f2000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec112180 con 0x7f39ec101de0 2024-09-16T02:04:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39e2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39ec102790 0x7f39ec19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39e2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39ec102790 0x7f39ec19ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55870/0 (socket says 172.21.15.103:55870) 2024-09-16T02:04:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39e2a00640 1 -- 172.21.15.103:0/395126672 learned_addr learned my addr 172.21.15.103:0/395126672 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39eb400640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39ec101de0 0x7f39ec19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.283+0000 7f39ebe00640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39ec106f10 0x7f39ec19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.284+0000 7f39eb400640 1 -- 172.21.15.103:0/395126672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39ec106f10 msgr2=0x7f39ec19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.284+0000 7f39eb400640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39ec106f10 0x7f39ec19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.284+0000 7f39eb400640 1 -- 172.21.15.103:0/395126672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39ec102790 msgr2=0x7f39ec19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.284+0000 7f39eb400640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39ec102790 0x7f39ec19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.284+0000 7f39eb400640 1 -- 172.21.15.103:0/395126672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ec19f680 con 0x7f39ec101de0 2024-09-16T02:04:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.284+0000 7f39eb400640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39ec101de0 0x7f39ec19a6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00e9f0 tx=0x7f39dc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.284+0000 7f39e8c00640 1 -- 172.21.15.103:0/395126672 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00ccc0 con 0x7f39ec101de0 2024-09-16T02:04:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.284+0000 7f39f2000640 1 -- 172.21.15.103:0/395126672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ec19f970 con 0x7f39ec101de0 2024-09-16T02:04:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.284+0000 7f39e8c00640 1 -- 172.21.15.103:0/395126672 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39dc005020 con 0x7f39ec101de0 2024-09-16T02:04:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.284+0000 7f39e8c00640 1 -- 172.21.15.103:0/395126672 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0049e0 con 0x7f39ec101de0 2024-09-16T02:04:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.285+0000 7f39f2000640 1 -- 172.21.15.103:0/395126672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ec1a7250 con 0x7f39ec101de0 2024-09-16T02:04:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.286+0000 7f39f2000640 1 -- 172.21.15.103:0/395126672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39ec1021c0 con 0x7f39ec101de0 2024-09-16T02:04:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.288+0000 7f39e8c00640 1 -- 172.21.15.103:0/395126672 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f39dc005560 con 0x7f39ec101de0 2024-09-16T02:04:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.289+0000 7f39e8c00640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f39d00a32d0 0x7f39d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.289+0000 7f39e8c00640 1 -- 172.21.15.103:0/395126672 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f39dc0cfd40 con 0x7f39ec101de0 2024-09-16T02:04:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.289+0000 7f39e2a00640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f39d00a32d0 0x7f39d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.290+0000 7f39e2a00640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f39d00a32d0 0x7f39d00a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39d40096c0 tx=0x7f39d40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.291+0000 7f39e8c00640 1 -- 172.21.15.103:0/395126672 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39dc0051d0 con 0x7f39ec101de0 2024-09-16T02:04:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.467+0000 7f39f2000640 1 -- 172.21.15.103:0/395126672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39ec1a7450 con 0x7f39ec101de0 2024-09-16T02:04:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.469+0000 7f39e8c00640 1 -- 172.21.15.103:0/395126672 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22861 (secure 0 0 0) 0x7f39dc09c890 con 0x7f39ec101de0 2024-09-16T02:04:54.471 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:54.471 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":79,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:04:29.624699+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":10,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":7,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:04:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.471+0000 7f39f2000640 1 -- 172.21.15.103:0/395126672 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f39d00a32d0 msgr2=0x7f39d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.471+0000 7f39f2000640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f39d00a32d0 0x7f39d00a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39d40096c0 tx=0x7f39d40046c0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.471+0000 7f39f2000640 1 -- 172.21.15.103:0/395126672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39ec101de0 msgr2=0x7f39ec19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.471+0000 7f39f2000640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39ec101de0 0x7f39ec19a6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00e9f0 tx=0x7f39dc00eeb0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.472+0000 7f39f2000640 1 -- 172.21.15.103:0/395126672 shutdown_connections 2024-09-16T02:04:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.472+0000 7f39f2000640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f39d00a32d0 0x7f39d00a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.472+0000 7f39f2000640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39ec106f10 0x7f39ec19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.472+0000 7f39f2000640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39ec102790 0x7f39ec19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.472+0000 7f39f2000640 1 --2- 172.21.15.103:0/395126672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39ec101de0 0x7f39ec19a6d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:54.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.472+0000 7f39f2000640 1 -- 172.21.15.103:0/395126672 >> 172.21.15.103:0/395126672 conn(0x7f39ec0fd8c0 msgr2=0x7f39ec0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:54.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.472+0000 7f39f2000640 1 -- 172.21.15.103:0/395126672 shutdown_connections 2024-09-16T02:04:54.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:54.472+0000 7f39f2000640 1 -- 172.21.15.103:0/395126672 wait complete. 2024-09-16T02:04:54.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:54 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/395126672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:54.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:54 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/395126672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:55.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:54 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/395126672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:04:55.114 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:04:55.479 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:55.794 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:55 smithi103 ceph-mon[30398]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:04:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:55 smithi172 ceph-mon[35340]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:04:56.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:55 smithi136 ceph-mon[35323]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:04:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.023+0000 7f4e81400640 1 -- 172.21.15.103:0/3717279683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c1049c0 msgr2=0x7f4e7c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.023+0000 7f4e81400640 1 --2- 172.21.15.103:0/3717279683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c1049c0 0x7f4e7c108a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009900 tx=0x7f4e6802a3f0 comp rx=0 tx=0).stop 2024-09-16T02:04:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.023+0000 7f4e81400640 1 -- 172.21.15.103:0/3717279683 shutdown_connections 2024-09-16T02:04:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.023+0000 7f4e81400640 1 --2- 172.21.15.103:0/3717279683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e7c109140 0x7f4e7c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.023+0000 7f4e81400640 1 --2- 172.21.15.103:0/3717279683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c1049c0 0x7f4e7c108a10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.023+0000 7f4e81400640 1 --2- 172.21.15.103:0/3717279683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7c104010 0x7f4e7c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.023+0000 7f4e81400640 1 -- 172.21.15.103:0/3717279683 >> 172.21.15.103:0/3717279683 conn(0x7f4e7c0ffa50 msgr2=0x7f4e7c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.023+0000 7f4e81400640 1 -- 172.21.15.103:0/3717279683 shutdown_connections 2024-09-16T02:04:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.023+0000 7f4e81400640 1 -- 172.21.15.103:0/3717279683 wait complete. 2024-09-16T02:04:56.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.024+0000 7f4e81400640 1 Processor -- start 2024-09-16T02:04:56.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.024+0000 7f4e81400640 1 -- start start 2024-09-16T02:04:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e81400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7c104010 0x7f4e7c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e81400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e7c1049c0 0x7f4e7c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e81400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c109140 0x7f4e7c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e81400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c114160 con 0x7f4e7c1049c0 2024-09-16T02:04:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e81400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c113fe0 con 0x7f4e7c104010 2024-09-16T02:04:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e81400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1142e0 con 0x7f4e7c109140 2024-09-16T02:04:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e79e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e7c1049c0 0x7f4e7c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e79e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e7c1049c0 0x7f4e7c19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55890/0 (socket says 172.21.15.103:55890) 2024-09-16T02:04:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e79e00640 1 -- 172.21.15.103:0/2637509670 learned_addr learned my addr 172.21.15.103:0/2637509670 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e7a800640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7c104010 0x7f4e7c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e7b200640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c109140 0x7f4e7c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e79e00640 1 -- 172.21.15.103:0/2637509670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c109140 msgr2=0x7f4e7c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e79e00640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c109140 0x7f4e7c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e79e00640 1 -- 172.21.15.103:0/2637509670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7c104010 msgr2=0x7f4e7c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e79e00640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7c104010 0x7f4e7c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.025+0000 7f4e79e00640 1 -- 172.21.15.103:0/2637509670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c19f610 con 0x7f4e7c1049c0 2024-09-16T02:04:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.026+0000 7f4e79e00640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e7c1049c0 0x7f4e7c19aba0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006010 tx=0x7f4e68004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.026+0000 7f4e73400640 1 -- 172.21.15.103:0/2637509670 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6803f030 con 0x7f4e7c1049c0 2024-09-16T02:04:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.026+0000 7f4e73400640 1 -- 172.21.15.103:0/2637509670 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e68004440 con 0x7f4e7c1049c0 2024-09-16T02:04:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.026+0000 7f4e81400640 1 -- 172.21.15.103:0/2637509670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c19f8a0 con 0x7f4e7c1049c0 2024-09-16T02:04:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.026+0000 7f4e73400640 1 -- 172.21.15.103:0/2637509670 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68035070 con 0x7f4e7c1049c0 2024-09-16T02:04:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.026+0000 7f4e81400640 1 -- 172.21.15.103:0/2637509670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c1a71e0 con 0x7f4e7c1049c0 2024-09-16T02:04:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.027+0000 7f4e81400640 1 -- 172.21.15.103:0/2637509670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7c1054f0 con 0x7f4e7c1049c0 2024-09-16T02:04:56.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.029+0000 7f4e73400640 1 -- 172.21.15.103:0/2637509670 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4e68042030 con 0x7f4e7c1049c0 2024-09-16T02:04:56.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.030+0000 7f4e73400640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4e600a32d0 0x7f4e600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:56.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.031+0000 7f4e73400640 1 -- 172.21.15.103:0/2637509670 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f4e680efe60 con 0x7f4e7c1049c0 2024-09-16T02:04:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.031+0000 7f4e7a800640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4e600a32d0 0x7f4e600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.031+0000 7f4e7a800640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4e600a32d0 0x7f4e600a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009bd0 tx=0x7f4e6c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:56.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.035+0000 7f4e73400640 1 -- 172.21.15.103:0/2637509670 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e680bc360 con 0x7f4e7c1049c0 2024-09-16T02:04:56.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.244+0000 7f4e81400640 1 -- 172.21.15.103:0/2637509670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e7c10c380 con 0x7f4e7c1049c0 2024-09-16T02:04:56.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.245+0000 7f4e73400640 1 -- 172.21.15.103:0/2637509670 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7f4e680c0660 con 0x7f4e7c1049c0 2024-09-16T02:04:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:04:56.246 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":23,"btime":"2024-09-16T02:03:50:098205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:03:49.812580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24505,24506,24545]},"id":1}]} 2024-09-16T02:04:56.246 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 23 2024-09-16T02:04:56.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 -- 172.21.15.103:0/2637509670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4e600a32d0 msgr2=0x7f4e600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:56.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4e600a32d0 0x7f4e600a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009bd0 tx=0x7f4e6c005f90 comp rx=0 tx=0).stop 2024-09-16T02:04:56.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 -- 172.21.15.103:0/2637509670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e7c1049c0 msgr2=0x7f4e7c19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:56.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e7c1049c0 0x7f4e7c19aba0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006010 tx=0x7f4e68004040 comp rx=0 tx=0).stop 2024-09-16T02:04:56.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 -- 172.21.15.103:0/2637509670 shutdown_connections 2024-09-16T02:04:56.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4e600a32d0 0x7f4e600a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:56.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c109140 0x7f4e7c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:56.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e7c1049c0 0x7f4e7c19aba0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:56.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 --2- 172.21.15.103:0/2637509670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7c104010 0x7f4e7c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:56.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 -- 172.21.15.103:0/2637509670 >> 172.21.15.103:0/2637509670 conn(0x7f4e7c0ffa50 msgr2=0x7f4e7c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:56.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.247+0000 7f4e81400640 1 -- 172.21.15.103:0/2637509670 shutdown_connections 2024-09-16T02:04:56.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:56.248+0000 7f4e81400640 1 -- 172.21.15.103:0/2637509670 wait complete. 2024-09-16T02:04:56.796 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:56 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2637509670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:56.932 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:56 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2637509670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:56.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T02:03:37.830568+0000', 'modified': '2024-09-16T02:03:49.812580+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24545, 'mds_1': 24506, 'mds_2': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/2566717089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 2566717089}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 2566717089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/2720370264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 2720370264}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2720370264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6837/1584538248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 1584538248}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 1584538248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.172:6839/3474994563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 3474994563}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 3474994563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.136:6837/236571207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 236571207}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 236571207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.136:6839/3041647128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 3041647128}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 3041647128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24505, 24506, 24545]} 2024-09-16T02:04:56.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-16T02:04:56.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T02:04:57.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:56 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2637509670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:04:57.355 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:57.594 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:57 smithi103 ceph-mon[30398]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:04:57.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:57 smithi172 ceph-mon[35340]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:04:57.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.871+0000 7f0b5a200640 1 -- 172.21.15.103:0/3170167314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b54101de0 msgr2=0x7f0b541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:57.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.871+0000 7f0b5a200640 1 --2- 172.21.15.103:0/3170167314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b54101de0 0x7f0b541021c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009ca0 tx=0x7f0b4402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.871+0000 7f0b5a200640 1 -- 172.21.15.103:0/3170167314 shutdown_connections 2024-09-16T02:04:57.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.871+0000 7f0b5a200640 1 --2- 172.21.15.103:0/3170167314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b54106f10 0x7f0b5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.871+0000 7f0b5a200640 1 --2- 172.21.15.103:0/3170167314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b54102790 0x7f0b541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.871+0000 7f0b5a200640 1 --2- 172.21.15.103:0/3170167314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b54101de0 0x7f0b541021c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.871+0000 7f0b5a200640 1 -- 172.21.15.103:0/3170167314 >> 172.21.15.103:0/3170167314 conn(0x7f0b540fd8c0 msgr2=0x7f0b540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:57.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.871+0000 7f0b5a200640 1 -- 172.21.15.103:0/3170167314 shutdown_connections 2024-09-16T02:04:57.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.872+0000 7f0b5a200640 1 -- 172.21.15.103:0/3170167314 wait complete. 2024-09-16T02:04:57.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.873+0000 7f0b5a200640 1 Processor -- start 2024-09-16T02:04:57.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b5a200640 1 -- start start 2024-09-16T02:04:57.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b5a200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b54101de0 0x7f0b5419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b5a200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b54102790 0x7f0b5419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b5a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b54106f10 0x7f0b5419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b5a200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b54111fc0 con 0x7f0b54101de0 2024-09-16T02:04:57.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b5a200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b54111e40 con 0x7f0b54102790 2024-09-16T02:04:57.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b5a200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b54112140 con 0x7f0b54106f10 2024-09-16T02:04:57.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b53400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b54101de0 0x7f0b5419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b53400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b54101de0 0x7f0b5419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55900/0 (socket says 172.21.15.103:55900) 2024-09-16T02:04:57.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b53400640 1 -- 172.21.15.103:0/3569741655 learned_addr learned my addr 172.21.15.103:0/3569741655 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b52a00640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b54102790 0x7f0b5419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.874+0000 7f0b53e00640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b54106f10 0x7f0b5419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.875+0000 7f0b53400640 1 -- 172.21.15.103:0/3569741655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b54106f10 msgr2=0x7f0b5419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.875+0000 7f0b53400640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b54106f10 0x7f0b5419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.875+0000 7f0b53400640 1 -- 172.21.15.103:0/3569741655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b54102790 msgr2=0x7f0b5419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.875+0000 7f0b53400640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b54102790 0x7f0b5419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.875+0000 7f0b53400640 1 -- 172.21.15.103:0/3569741655 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5419f650 con 0x7f0b54101de0 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.875+0000 7f0b53400640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b54101de0 0x7f0b5419a6a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0b44005e30 tx=0x7f0b44003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.875+0000 7f0b4be00640 1 -- 172.21.15.103:0/3569741655 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b440316d0 con 0x7f0b54101de0 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.875+0000 7f0b4be00640 1 -- 172.21.15.103:0/3569741655 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b44031870 con 0x7f0b54101de0 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.875+0000 7f0b5a200640 1 -- 172.21.15.103:0/3569741655 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5419f8e0 con 0x7f0b54101de0 2024-09-16T02:04:57.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.875+0000 7f0b4be00640 1 -- 172.21.15.103:0/3569741655 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4402fb90 con 0x7f0b54101de0 2024-09-16T02:04:57.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.876+0000 7f0b5a200640 1 -- 172.21.15.103:0/3569741655 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b541a7220 con 0x7f0b54101de0 2024-09-16T02:04:57.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.878+0000 7f0b4be00640 1 -- 172.21.15.103:0/3569741655 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0b4402fd50 con 0x7f0b54101de0 2024-09-16T02:04:57.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.878+0000 7f0b5cc38740 1 Processor -- start 2024-09-16T02:04:57.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b5cc38740 1 -- start start 2024-09-16T02:04:57.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78a2c50 0x561cb78acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78af900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561cb78080b0 con 0x561cb78ad510 2024-09-16T02:04:57.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cb77afab0 con 0x561cb78a2c50 2024-09-16T02:04:57.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561cb7804070 con 0x561cb78a2330 2024-09-16T02:04:57.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b53e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b52a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b53e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:49312/0 (socket says 172.21.15.103:49312) 2024-09-16T02:04:57.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b53e00640 1 -- 172.21.15.103:0/3841676682 learned_addr learned my addr 172.21.15.103:0/3841676682 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:57.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.879+0000 7f0b53400640 1 --2- 172.21.15.103:0/3841676682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78a2c50 0x561cb78acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.880+0000 7f0b52a00640 1 -- 172.21.15.103:0/3841676682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 msgr2=0x561cb78a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:57.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.880+0000 7f0b52a00640 1 --2- 172.21.15.103:0/3841676682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.880+0000 7f0b52a00640 1 -- 172.21.15.103:0/3841676682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78a2c50 msgr2=0x561cb78acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:57.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.880+0000 7f0b52a00640 1 --2- 172.21.15.103:0/3841676682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78a2c50 0x561cb78acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.880+0000 7f0b52a00640 1 -- 172.21.15.103:0/3841676682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb77bc4a0 con 0x561cb78ad510 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.880+0000 7f0b53e00640 1 --2- 172.21.15.103:0/3841676682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.880+0000 7f0b4be00640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0b380a3350 0x7f0b380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.880+0000 7f0b52a00640 1 --2- 172.21.15.103:0/3841676682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78af900 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000bbc0 tx=0x7f0b400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff8d292b1ae8a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.880+0000 7f0b4be00640 1 -- 172.21.15.103:0/3569741655 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f0b440f1060 con 0x7f0b54101de0 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.881+0000 7f0b53e00640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0b380a3350 0x7f0b380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.881+0000 7f0b3fe00640 1 -- 172.21.15.103:0/3841676682 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40004030 con 0x561cb78ad510 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.881+0000 7f0b3fe00640 1 -- 172.21.15.103:0/3841676682 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b400041d0 con 0x561cb78ad510 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.881+0000 7f0b3fe00640 1 -- 172.21.15.103:0/3841676682 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b400116a0 con 0x561cb78ad510 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.881+0000 7f0b5cc38740 1 -- 172.21.15.103:0/3841676682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 msgr2=0x561cb78af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.881+0000 7f0b53e00640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0b380a3350 0x7f0b380a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00b990 tx=0x7f0b4c00b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:57.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.881+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/3841676682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78af900 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000bbc0 tx=0x7f0b400107a0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.882+0000 7f0b5cc38740 1 -- 172.21.15.103:0/3841676682 shutdown_connections 2024-09-16T02:04:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.882+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/3841676682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78af900 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.882+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/3841676682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78a2c50 0x561cb78acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.882+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/3841676682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.882+0000 7f0b5cc38740 1 -- 172.21.15.103:0/3841676682 >> 172.21.15.103:0/3841676682 conn(0x561cb789e8c0 msgr2=0x561cb789eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.882+0000 7f0b5cc38740 1 -- 172.21.15.103:0/3841676682 shutdown_connections 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.882+0000 7f0b5cc38740 1 -- 172.21.15.103:0/3841676682 wait complete. 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.882+0000 7f0b5cc38740 1 -- start start 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.882+0000 7f0b5cc38740 1 Processor -- start 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.883+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78a2330 0x561cb78aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.883+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2c50 0x561cb78aab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.883+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78ab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.883+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561cb78b0060 con 0x561cb78ad510 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.883+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cb7790920 con 0x561cb78a2330 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.883+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561cb78b01e0 con 0x561cb78a2c50 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.883+0000 7f0b5cc38740 10 client.0 _ll_register_callbacks cb 0x561cb77b48a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.883+0000 7f0b53e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.883+0000 7f0b53e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55910/0 (socket says 172.21.15.103:55910) 2024-09-16T02:04:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.883+0000 7f0b53e00640 1 -- 172.21.15.103:0/1950731117 learned_addr learned my addr 172.21.15.103:0/1950731117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.884+0000 7f0b53e00640 1 -- 172.21.15.103:0/1950731117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2c50 msgr2=0x561cb78aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.884+0000 7f0b53e00640 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2c50 0x561cb78aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.884+0000 7f0b53e00640 1 -- 172.21.15.103:0/1950731117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78a2330 msgr2=0x561cb78aa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.884+0000 7f0b53e00640 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78a2330 0x561cb78aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.884+0000 7f0b53e00640 1 -- 172.21.15.103:0/1950731117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb75792b0 con 0x561cb78ad510 2024-09-16T02:04:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.884+0000 7f0b53e00640 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78ab070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c003c20 tx=0x7f0b4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1245cb7a7f4cd54e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.884+0000 7f0b3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:04:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.884+0000 7f0b3cc00640 1 -- 172.21.15.103:0/1950731117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c02c060 con 0x561cb78ad510 2024-09-16T02:04:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.884+0000 7f0b3cc00640 1 -- 172.21.15.103:0/1950731117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f0b4c017030 con 0x561cb78ad510 2024-09-16T02:04:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.884+0000 7f0b5cc38740 1 -- 172.21.15.103:0/1950731117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b400041d0 con 0x561cb78ad510 2024-09-16T02:04:57.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b3cc00640 1 -- 172.21.15.103:0/1950731117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0b4c003e60 con 0x561cb78ad510 2024-09-16T02:04:57.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b5cc38740 10 client.14880 fetch_fsmap learned FSMap version 23 2024-09-16T02:04:57.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b5cc38740 1 -- 172.21.15.103:0/1950731117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561cb78bbf20 con 0x561cb78ad510 2024-09-16T02:04:57.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b3cc00640 1 -- 172.21.15.103:0/1950731117 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b400041d0 con 0x561cb78ad510 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b3cc00640 1 -- 172.21.15.103:0/1950731117 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f0b4c0116a0 con 0x561cb78ad510 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b5cc38740 10 client.14880 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b5cc38740 10 client.14880 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b5cc38740 20 client.14880 populate_metadata read hostname 'smithi103' 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb78be690 0x561cb78bfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b5cc38740 4 client.14880 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.885+0000 7f0b5cc38740 1 -- 172.21.15.103:0/1950731117 --> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cb7815620 con 0x561cb78be690 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.886+0000 7f0b53400640 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb78be690 0x561cb78bfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.886+0000 7f0b53400640 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb78be690 0x561cb78bfb00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94a492eaaec31cf2 server_cookie=e3a24fb05bb5d481 in_seq=0 out_seq=0 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.886+0000 7f0b3cc00640 10 client.14880 ms_handle_connect on v2:172.21.15.172:6838/3474994563 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b3cc00640 1 -- 172.21.15.103:0/1950731117 <== mds.2 v2:172.21.15.172:6838/3474994563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cb7815620 con 0x561cb78be690 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b3cc00640 10 client.14880 handle_command_reply: tid=0 2024-09-16T02:04:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 client.14880 shutdown 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 -- 172.21.15.103:0/1950731117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 msgr2=0x561cb78ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78ab070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c003c20 tx=0x7f0b4c004730 comp rx=0 tx=0).stop 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 -- 172.21.15.103:0/1950731117 shutdown_connections 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb78be690 0x561cb78bfb00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78ad510 0x561cb78ab070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2c50 0x561cb78aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/1950731117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78a2330 0x561cb78aa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 -- 172.21.15.103:0/1950731117 >> 172.21.15.103:0/1950731117 conn(0x561cb789e8c0 msgr2=0x561cb78a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b3cc00640 0 client.14880 ms_handle_reset on v2:172.21.15.172:6838/3474994563 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 -- 172.21.15.103:0/1950731117 shutdown_connections 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.887+0000 7f0b5cc38740 1 -- 172.21.15.103:0/1950731117 wait complete. 2024-09-16T02:04:57.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.888+0000 7f0b5cc38740 20 client.14880 trim_cache size 0 max 16384 2024-09-16T02:04:57.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.906+0000 7f0b5cc38740 1 Processor -- start 2024-09-16T02:04:57.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.906+0000 7f0b5cc38740 1 -- start start 2024-09-16T02:04:57.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.906+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.906+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.906+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 0x561cb78a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.906+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561cb78bde40 con 0x561cb78a2c50 2024-09-16T02:04:57.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.906+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cb78be510 con 0x561cb78ad510 2024-09-16T02:04:57.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561cb78bebe0 con 0x561cb78a2330 2024-09-16T02:04:57.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b52a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b52a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55922/0 (socket says 172.21.15.103:55922) 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b52a00640 1 -- 172.21.15.103:0/3956915207 learned_addr learned my addr 172.21.15.103:0/3956915207 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b53400640 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b53e00640 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 0x561cb78a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b53400640 1 -- 172.21.15.103:0/3956915207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 msgr2=0x561cb78a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b53400640 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 0x561cb78a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b53400640 1 -- 172.21.15.103:0/3956915207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 msgr2=0x561cb78a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b53400640 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b53400640 1 -- 172.21.15.103:0/3956915207 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb78a2120 con 0x561cb78a2330 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b52a00640 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b53e00640 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 0x561cb78a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.907+0000 7f0b53400640 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a9eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b440eeb10 tx=0x7f0b440095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd4d938aab279ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.908+0000 7f0b3f400640 1 -- 172.21.15.103:0/3956915207 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b440bd3f0 con 0x561cb78a2330 2024-09-16T02:04:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.908+0000 7f0b3f400640 1 -- 172.21.15.103:0/3956915207 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b44036030 con 0x561cb78a2330 2024-09-16T02:04:57.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.908+0000 7f0b3f400640 1 -- 172.21.15.103:0/3956915207 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b440bcf80 con 0x561cb78a2330 2024-09-16T02:04:57.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.908+0000 7f0b5cc38740 1 -- 172.21.15.103:0/3956915207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 msgr2=0x561cb78a9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:57.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.908+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a9eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b440eeb10 tx=0x7f0b440095c0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.908+0000 7f0b5cc38740 1 -- 172.21.15.103:0/3956915207 shutdown_connections 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.908+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 0x561cb78a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.908+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.908+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/3956915207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a9eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.908+0000 7f0b5cc38740 1 -- 172.21.15.103:0/3956915207 >> 172.21.15.103:0/3956915207 conn(0x561cb789e8c0 msgr2=0x561cb78a4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.909+0000 7f0b5cc38740 1 -- 172.21.15.103:0/3956915207 shutdown_connections 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.909+0000 7f0b5cc38740 1 -- 172.21.15.103:0/3956915207 wait complete. 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.909+0000 7f0b5cc38740 1 -- start start 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.909+0000 7f0b5cc38740 1 Processor -- start 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b5cc38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 0x561cb7926d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561cb78bf980 con 0x561cb78a2c50 2024-09-16T02:04:57.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cb78bf2b0 con 0x561cb78ad510 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b5cc38740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561cb78c0050 con 0x561cb78a2330 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b5cc38740 10 client.0 _ll_register_callbacks cb 0x7f0b4c004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b52a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b52a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55936/0 (socket says 172.21.15.103:55936) 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b52a00640 1 -- 172.21.15.103:0/2674591719 learned_addr learned my addr 172.21.15.103:0/2674591719 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b53400640 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b53e00640 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 0x561cb7926d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b53400640 1 -- 172.21.15.103:0/2674591719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 msgr2=0x561cb7926d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b53400640 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 0x561cb7926d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b53400640 1 -- 172.21.15.103:0/2674591719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 msgr2=0x561cb78a7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:57.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b53400640 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.910+0000 7f0b53400640 1 -- 172.21.15.103:0/2674591719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb79272d0 con 0x561cb78a2330 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b53400640 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a7050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b440bd7e0 tx=0x7f0b440bbb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b220bda5be8587e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b3cc00640 1 -- 172.21.15.103:0/2674591719 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b440bd160 con 0x561cb78a2330 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b3cc00640 1 -- 172.21.15.103:0/2674591719 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f0b440b9c90 con 0x561cb78a2330 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b5cc38740 1 -- 172.21.15.103:0/2674591719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b44036030 con 0x561cb78a2330 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b3cc00640 1 -- 172.21.15.103:0/2674591719 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0b440bd620 con 0x561cb78a2330 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b5cc38740 10 client.24856 fetch_fsmap learned FSMap version 23 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b5cc38740 1 -- 172.21.15.103:0/2674591719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561cb7928430 con 0x561cb78a2330 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b3cc00640 1 -- 172.21.15.103:0/2674591719 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b44036030 con 0x561cb78a2330 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b3cc00640 1 -- 172.21.15.103:0/2674591719 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f0b440f30d0 con 0x561cb78a2330 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.911+0000 7f0b5cc38740 10 client.24856 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.912+0000 7f0b5cc38740 10 client.24856 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.912+0000 7f0b5cc38740 20 client.24856 populate_metadata read hostname 'smithi103' 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.912+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb792cce0 0x561cb792b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.912+0000 7f0b5cc38740 4 client.24856 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.912+0000 7f0b5cc38740 1 -- 172.21.15.103:0/2674591719 --> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] -- command(tid 0: {"prefix": "status"}) -- 0x561cb780fb30 con 0x561cb792cce0 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.912+0000 7f0b52a00640 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb792cce0 0x561cb792b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.912+0000 7f0b52a00640 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb792cce0 0x561cb792b9f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c6102a7b868d674 server_cookie=f8e2640d4aa7a5d5 in_seq=0 out_seq=0 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.912+0000 7f0b3cc00640 10 client.24856 ms_handle_connect on v2:172.21.15.172:6838/3474994563 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b3cc00640 1 -- 172.21.15.103:0/2674591719 <== mds.2 v2:172.21.15.172:6838/3474994563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561cb780fb30 con 0x561cb792cce0 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b3cc00640 10 client.24856 handle_command_reply: tid=0 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 client.24856 shutdown 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 -- 172.21.15.103:0/2674591719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 msgr2=0x561cb78a7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a7050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b440bd7e0 tx=0x7f0b440bbb30 comp rx=0 tx=0).stop 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 -- 172.21.15.103:0/2674591719 shutdown_connections 2024-09-16T02:04:57.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb792cce0 0x561cb792b9f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb78ad510 0x561cb7926d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b3cc00640 0 client.24856 ms_handle_reset on v2:172.21.15.172:6838/3474994563 2024-09-16T02:04:57.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb78a2c50 0x561cb78a7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 --2- 172.21.15.103:0/2674591719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb78a2330 0x561cb78a7050 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 -- 172.21.15.103:0/2674591719 >> 172.21.15.103:0/2674591719 conn(0x561cb789e8c0 msgr2=0x561cb78bd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:57.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 -- 172.21.15.103:0/2674591719 shutdown_connections 2024-09-16T02:04:57.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 1 -- 172.21.15.103:0/2674591719 wait complete. 2024-09-16T02:04:57.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.913+0000 7f0b5cc38740 20 client.24856 trim_cache size 0 max 16384 2024-09-16T02:04:57.919 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:04:57.919 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "a20dd7a4-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24505, 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 70.823054175999999, 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 23, 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 79, 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 77, 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 102.205765982 2024-09-16T02:04:57.920 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:04:57.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 -- 172.21.15.103:0/3569741655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0b380a3350 msgr2=0x7f0b380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:57.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0b380a3350 0x7f0b380a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00b990 tx=0x7f0b4c00b2e0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 -- 172.21.15.103:0/3569741655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b54101de0 msgr2=0x7f0b5419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:57.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b54101de0 0x7f0b5419a6a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0b44005e30 tx=0x7f0b44003f90 comp rx=0 tx=0).stop 2024-09-16T02:04:57.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 -- 172.21.15.103:0/3569741655 shutdown_connections 2024-09-16T02:04:57.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0b380a3350 0x7f0b380a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b54106f10 0x7f0b5419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b54102790 0x7f0b5419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 --2- 172.21.15.103:0/3569741655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b54101de0 0x7f0b5419a6a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:57.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 -- 172.21.15.103:0/3569741655 >> 172.21.15.103:0/3569741655 conn(0x7f0b540fd8c0 msgr2=0x7f0b540feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:57.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 -- 172.21.15.103:0/3569741655 shutdown_connections 2024-09-16T02:04:57.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:57.914+0000 7f0b5a200640 1 -- 172.21.15.103:0/3569741655 wait complete. 2024-09-16T02:04:58.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:57 smithi136 ceph-mon[35323]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:04:58.644 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T02:04:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:04:59.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.504+0000 7f6f23200640 1 -- 172.21.15.103:0/3694788795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c102790 msgr2=0x7f6f1c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:59.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.504+0000 7f6f23200640 1 --2- 172.21.15.103:0/3694788795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c102790 0x7f6f1c1067e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009900 tx=0x7f6f0802a410 comp rx=0 tx=0).stop 2024-09-16T02:04:59.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.504+0000 7f6f23200640 1 -- 172.21.15.103:0/3694788795 shutdown_connections 2024-09-16T02:04:59.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.504+0000 7f6f23200640 1 --2- 172.21.15.103:0/3694788795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c106f10 0x7f6f1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.504+0000 7f6f23200640 1 --2- 172.21.15.103:0/3694788795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c102790 0x7f6f1c1067e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.504+0000 7f6f23200640 1 --2- 172.21.15.103:0/3694788795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f1c101de0 0x7f6f1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.504+0000 7f6f23200640 1 -- 172.21.15.103:0/3694788795 >> 172.21.15.103:0/3694788795 conn(0x7f6f1c0fd8c0 msgr2=0x7f6f1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:59.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.504+0000 7f6f23200640 1 -- 172.21.15.103:0/3694788795 shutdown_connections 2024-09-16T02:04:59.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.504+0000 7f6f23200640 1 -- 172.21.15.103:0/3694788795 wait complete. 2024-09-16T02:04:59.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.505+0000 7f6f23200640 1 Processor -- start 2024-09-16T02:04:59.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.505+0000 7f6f23200640 1 -- start start 2024-09-16T02:04:59.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.505+0000 7f6f23200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c101de0 0x7f6f1c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.505+0000 7f6f23200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c102790 0x7f6f1c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.505+0000 7f6f23200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f1c106f10 0x7f6f1c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.505+0000 7f6f23200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c111fc0 con 0x7f6f1c101de0 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.505+0000 7f6f23200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c111e40 con 0x7f6f1c106f10 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.505+0000 7f6f23200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c112140 con 0x7f6f1c102790 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f1be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c101de0 0x7f6f1c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f21200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f1c106f10 0x7f6f1c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f21200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f1c106f10 0x7f6f1c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:44442/0 (socket says 172.21.15.103:44442) 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f21200640 1 -- 172.21.15.103:0/3434548594 learned_addr learned my addr 172.21.15.103:0/3434548594 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f1b400640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c102790 0x7f6f1c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f1be00640 1 -- 172.21.15.103:0/3434548594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c102790 msgr2=0x7f6f1c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f1be00640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c102790 0x7f6f1c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f1be00640 1 -- 172.21.15.103:0/3434548594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f1c106f10 msgr2=0x7f6f1c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f1be00640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f1c106f10 0x7f6f1c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f1be00640 1 -- 172.21.15.103:0/3434548594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c1a6920 con 0x7f6f1c101de0 2024-09-16T02:04:59.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f21200640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f1c106f10 0x7f6f1c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:59.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.506+0000 7f6f1be00640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c101de0 0x7f6f1c0744f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00d6d0 tx=0x7f6f0c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:59.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.507+0000 7f6f18c00640 1 -- 172.21.15.103:0/3434548594 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c017070 con 0x7f6f1c101de0 2024-09-16T02:04:59.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.507+0000 7f6f18c00640 1 -- 172.21.15.103:0/3434548594 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0c009820 con 0x7f6f1c101de0 2024-09-16T02:04:59.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.507+0000 7f6f23200640 1 -- 172.21.15.103:0/3434548594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c1a6c10 con 0x7f6f1c101de0 2024-09-16T02:04:59.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.507+0000 7f6f18c00640 1 -- 172.21.15.103:0/3434548594 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c004d00 con 0x7f6f1c101de0 2024-09-16T02:04:59.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.507+0000 7f6f23200640 1 -- 172.21.15.103:0/3434548594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c1a7170 con 0x7f6f1c101de0 2024-09-16T02:04:59.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.510+0000 7f6f18c00640 1 -- 172.21.15.103:0/3434548594 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6f0c0099d0 con 0x7f6f1c101de0 2024-09-16T02:04:59.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.510+0000 7f6f25dbd740 1 Processor -- start 2024-09-16T02:04:59.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.510+0000 7f6f25dbd740 1 -- start start 2024-09-16T02:04:59.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.510+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.510+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223ac50 0x558802244fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.510+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 0x558802247900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5588021a00b0 con 0x55880223a330 2024-09-16T02:04:59.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558802147ab0 con 0x558802245510 2024-09-16T02:04:59.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55880219c070 con 0x55880223ac50 2024-09-16T02:04:59.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f21200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f21200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35276/0 (socket says 172.21.15.103:35276) 2024-09-16T02:04:59.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f21200640 1 -- 172.21.15.103:0/3933822794 learned_addr learned my addr 172.21.15.103:0/3933822794 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:59.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f1b400640 1 --2- 172.21.15.103:0/3933822794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 0x558802247900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f1be00640 1 --2- 172.21.15.103:0/3933822794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223ac50 0x558802244fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f21200640 1 -- 172.21.15.103:0/3933822794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223ac50 msgr2=0x558802244fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:59.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f21200640 1 --2- 172.21.15.103:0/3933822794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223ac50 0x558802244fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f21200640 1 -- 172.21.15.103:0/3933822794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 msgr2=0x558802247900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:59.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f21200640 1 --2- 172.21.15.103:0/3933822794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 0x558802247900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f21200640 1 -- 172.21.15.103:0/3933822794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588021544a0 con 0x55880223a330 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.511+0000 7f6f1b400640 1 --2- 172.21.15.103:0/3933822794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 0x558802247900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.512+0000 7f6f18c00640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f000a3350 0x7f6f000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.512+0000 7f6f21200640 1 --2- 172.21.15.103:0/3933822794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223a710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6f14011780 tx=0x7f6f14011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4faa95a1521cf70 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.512+0000 7f6f18c00640 1 -- 172.21.15.103:0/3434548594 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f6f0c0cf560 con 0x7f6f1c101de0 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.512+0000 7f6f1be00640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f000a3350 0x7f6f000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.512+0000 7f6f11600640 1 -- 172.21.15.103:0/3933822794 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400fb40 con 0x55880223a330 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.512+0000 7f6f11600640 1 -- 172.21.15.103:0/3933822794 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f14011e90 con 0x55880223a330 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.512+0000 7f6f11600640 1 -- 172.21.15.103:0/3933822794 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1401a650 con 0x55880223a330 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.513+0000 7f6f25dbd740 1 -- 172.21.15.103:0/3933822794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 msgr2=0x55880223a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.513+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/3933822794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223a710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6f14011780 tx=0x7f6f14011c40 comp rx=0 tx=0).stop 2024-09-16T02:04:59.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.513+0000 7f6f1be00640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f000a3350 0x7f6f000a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c098430 tx=0x7f6f0c0d2f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:59.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.513+0000 7f6f25dbd740 1 -- 172.21.15.103:0/3933822794 shutdown_connections 2024-09-16T02:04:59.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.513+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/3933822794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 0x558802247900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.513+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/3933822794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223ac50 0x558802244fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.513+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/3933822794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223a710 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.513+0000 7f6f25dbd740 1 -- 172.21.15.103:0/3933822794 >> 172.21.15.103:0/3933822794 conn(0x5588022368c0 msgr2=0x558802236ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:59.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.513+0000 7f6f25dbd740 1 -- 172.21.15.103:0/3933822794 shutdown_connections 2024-09-16T02:04:59.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.513+0000 7f6f25dbd740 1 -- 172.21.15.103:0/3933822794 wait complete. 2024-09-16T02:04:59.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.514+0000 7f6f25dbd740 1 -- start start 2024-09-16T02:04:59.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.514+0000 7f6f25dbd740 1 Processor -- start 2024-09-16T02:04:59.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.514+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x558802242510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.514+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55880223ac50 0x558802242a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.514+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558802245510 0x558802242f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.514+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558802248060 con 0x55880223a330 2024-09-16T02:04:59.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.514+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558802128920 con 0x55880223ac50 2024-09-16T02:04:59.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.514+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5588022481e0 con 0x558802245510 2024-09-16T02:04:59.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f25dbd740 10 client.0 _ll_register_callbacks cb 0x55880214c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:59.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f21200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x558802242510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f21200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x558802242510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35288/0 (socket says 172.21.15.103:35288) 2024-09-16T02:04:59.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f21200640 1 -- 172.21.15.103:0/2592711084 learned_addr learned my addr 172.21.15.103:0/2592711084 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:59.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f1b400640 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55880223ac50 0x558802242a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f21200640 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558802245510 0x558802242f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f21200640 1 -- 172.21.15.103:0/2592711084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558802245510 msgr2=0x558802242f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:59.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f21200640 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558802245510 0x558802242f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f21200640 1 -- 172.21.15.103:0/2592711084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55880223ac50 msgr2=0x558802242a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:59.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f21200640 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55880223ac50 0x558802242a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f21200640 1 -- 172.21.15.103:0/2592711084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558801f112b0 con 0x55880223a330 2024-09-16T02:04:59.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f21200640 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x558802242510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6f140092b0 tx=0x7f6f14012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f0991d49e924e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:59.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.515+0000 7f6f06000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:04:59.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.516+0000 7f6f06000640 1 -- 172.21.15.103:0/2592711084 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1402e060 con 0x55880223a330 2024-09-16T02:04:59.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.516+0000 7f6f06000640 1 -- 172.21.15.103:0/2592711084 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f6f1400fd50 con 0x55880223a330 2024-09-16T02:04:59.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.516+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2592711084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f14011e90 con 0x55880223a330 2024-09-16T02:04:59.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.516+0000 7f6f06000640 1 -- 172.21.15.103:0/2592711084 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6f14012450 con 0x55880223a330 2024-09-16T02:04:59.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.516+0000 7f6f25dbd740 10 client.14898 fetch_fsmap learned FSMap version 23 2024-09-16T02:04:59.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.516+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2592711084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558802253e10 con 0x55880223a330 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.516+0000 7f6f06000640 1 -- 172.21.15.103:0/2592711084 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f14011e90 con 0x55880223a330 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.517+0000 7f6f06000640 1 -- 172.21.15.103:0/2592711084 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6f1400f040 con 0x55880223a330 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.517+0000 7f6f25dbd740 10 client.14898 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.517+0000 7f6f25dbd740 10 client.14898 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.517+0000 7f6f25dbd740 20 client.14898 populate_metadata read hostname 'smithi103' 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.517+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x558802256070 0x5588022552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.517+0000 7f6f25dbd740 4 client.14898 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.517+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2592711084 --> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588021ad620 con 0x558802256070 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.517+0000 7f6f1b400640 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x558802256070 0x5588022552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.518+0000 7f6f1b400640 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x558802256070 0x5588022552d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95e7d43ef6a1fc85 server_cookie=967ab8dd4c152e6d in_seq=0 out_seq=0 2024-09-16T02:04:59.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.518+0000 7f6f06000640 10 client.14898 ms_handle_connect on v2:172.21.15.136:6836/236571207 2024-09-16T02:04:59.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.518+0000 7f6f06000640 1 -- 172.21.15.103:0/2592711084 <== mds.1 v2:172.21.15.136:6836/236571207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588021ad620 con 0x558802256070 2024-09-16T02:04:59.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f06000640 10 client.14898 handle_command_reply: tid=0 2024-09-16T02:04:59.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 client.14898 shutdown 2024-09-16T02:04:59.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2592711084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 msgr2=0x558802242510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x558802242510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6f140092b0 tx=0x7f6f14012e60 comp rx=0 tx=0).stop 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2592711084 shutdown_connections 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558802245510 0x558802242f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x558802256070 0x5588022552d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55880223ac50 0x558802242a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2592711084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x558802242510 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f06000640 0 client.14898 ms_handle_reset on v2:172.21.15.136:6836/236571207 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2592711084 >> 172.21.15.103:0/2592711084 conn(0x5588022368c0 msgr2=0x55880223c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2592711084 shutdown_connections 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2592711084 wait complete. 2024-09-16T02:04:59.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.519+0000 7f6f25dbd740 20 client.14898 trim_cache size 0 max 16384 2024-09-16T02:04:59.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.537+0000 7f6f25dbd740 1 Processor -- start 2024-09-16T02:04:59.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.537+0000 7f6f25dbd740 1 -- start start 2024-09-16T02:04:59.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223a330 0x55880223f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223ac50 0x55880223fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 0x55880223bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558802255820 con 0x55880223ac50 2024-09-16T02:04:59.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558802255ef0 con 0x558802245510 2024-09-16T02:04:59.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5588022565c0 con 0x55880223a330 2024-09-16T02:04:59.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f1b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223ac50 0x55880223fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f1b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223ac50 0x55880223fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35294/0 (socket says 172.21.15.103:35294) 2024-09-16T02:04:59.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f1b400640 1 -- 172.21.15.103:0/1935355117 learned_addr learned my addr 172.21.15.103:0/1935355117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:59.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f21200640 1 --2- 172.21.15.103:0/1935355117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223a330 0x55880223f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f21200640 1 --2- 172.21.15.103:0/1935355117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 0x55880223bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f1b400640 1 -- 172.21.15.103:0/1935355117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223a330 msgr2=0x55880223f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f1b400640 1 --2- 172.21.15.103:0/1935355117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223a330 0x55880223f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f1b400640 1 -- 172.21.15.103:0/1935355117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 msgr2=0x55880223bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f1b400640 1 --2- 172.21.15.103:0/1935355117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 0x55880223bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f1b400640 1 -- 172.21.15.103:0/1935355117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558802244e80 con 0x55880223ac50 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.538+0000 7f6f21200640 1 --2- 172.21.15.103:0/1935355117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223a330 0x55880223f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.539+0000 7f6f1b400640 1 --2- 172.21.15.103:0/1935355117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223ac50 0x55880223fcd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009230 tx=0x7f6f08005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84032727c62ecf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.539+0000 7f6f10c00640 1 -- 172.21.15.103:0/1935355117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08035040 con 0x55880223ac50 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.539+0000 7f6f10c00640 1 -- 172.21.15.103:0/1935355117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0804b070 con 0x55880223ac50 2024-09-16T02:04:59.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.539+0000 7f6f10c00640 1 -- 172.21.15.103:0/1935355117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08031910 con 0x55880223ac50 2024-09-16T02:04:59.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 -- 172.21.15.103:0/1935355117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223ac50 msgr2=0x55880223fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:59.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/1935355117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223ac50 0x55880223fcd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009230 tx=0x7f6f08005c00 comp rx=0 tx=0).stop 2024-09-16T02:04:59.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 -- 172.21.15.103:0/1935355117 shutdown_connections 2024-09-16T02:04:59.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/1935355117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558802245510 0x55880223bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/1935355117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223ac50 0x55880223fcd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/1935355117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55880223a330 0x55880223f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 -- 172.21.15.103:0/1935355117 >> 172.21.15.103:0/1935355117 conn(0x5588022368c0 msgr2=0x558802240de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 -- 172.21.15.103:0/1935355117 shutdown_connections 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 -- 172.21.15.103:0/1935355117 wait complete. 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 -- start start 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.540+0000 7f6f25dbd740 1 Processor -- start 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55880223ac50 0x55880223fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f25dbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558802245510 0x558802240060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558802257360 con 0x55880223a330 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558802256c90 con 0x55880223ac50 2024-09-16T02:04:59.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f25dbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558802257a30 con 0x558802245510 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f25dbd740 10 client.0 _ll_register_callbacks cb 0x7f6f14011600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f21200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f21200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35304/0 (socket says 172.21.15.103:35304) 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f21200640 1 -- 172.21.15.103:0/2596220322 learned_addr learned my addr 172.21.15.103:0/2596220322 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f1b400640 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55880223ac50 0x55880223fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f21200640 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558802245510 0x558802240060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f21200640 1 -- 172.21.15.103:0/2596220322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558802245510 msgr2=0x558802240060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f21200640 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558802245510 0x558802240060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f21200640 1 -- 172.21.15.103:0/2596220322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55880223ac50 msgr2=0x55880223fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f21200640 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55880223ac50 0x55880223fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.541+0000 7f6f21200640 1 -- 172.21.15.103:0/2596220322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55880225dcf0 con 0x55880223a330 2024-09-16T02:04:59.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.542+0000 7f6f21200640 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223f5e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6f140098b0 tx=0x7f6f14012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda5f5dd1ecceec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:04:59.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.542+0000 7f6f06000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:04:59.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.542+0000 7f6f06000640 1 -- 172.21.15.103:0/2596220322 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1402e040 con 0x55880223a330 2024-09-16T02:04:59.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.542+0000 7f6f06000640 1 -- 172.21.15.103:0/2596220322 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f6f140127f0 con 0x55880223a330 2024-09-16T02:04:59.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.542+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2596220322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f0804b070 con 0x55880223a330 2024-09-16T02:04:59.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.542+0000 7f6f06000640 1 -- 172.21.15.103:0/2596220322 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6f14011e90 con 0x55880223a330 2024-09-16T02:04:59.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.542+0000 7f6f25dbd740 10 client.14904 fetch_fsmap learned FSMap version 23 2024-09-16T02:04:59.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.542+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2596220322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5588022c0a10 con 0x55880223a330 2024-09-16T02:04:59.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.546+0000 7f6f06000640 1 -- 172.21.15.103:0/2596220322 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0804b070 con 0x55880223a330 2024-09-16T02:04:59.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.546+0000 7f6f06000640 1 -- 172.21.15.103:0/2596220322 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6f14026590 con 0x55880223a330 2024-09-16T02:04:59.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.546+0000 7f6f25dbd740 10 client.14904 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:04:59.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.546+0000 7f6f25dbd740 10 client.14904 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T02:04:59.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.546+0000 7f6f25dbd740 20 client.14904 populate_metadata read hostname 'smithi103' 2024-09-16T02:04:59.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.546+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x5588022c5530 0x5588022c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:04:59.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.546+0000 7f6f25dbd740 4 client.14904 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:04:59.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.546+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2596220322 --> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] -- command(tid 0: {"prefix": "status"}) -- 0x5588021a7b30 con 0x5588022c5530 2024-09-16T02:04:59.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.547+0000 7f6f1b400640 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x5588022c5530 0x5588022c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:04:59.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.547+0000 7f6f1b400640 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x5588022c5530 0x5588022c4240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c47b01ef3d8f361d server_cookie=fcc8dc035aa938f1 in_seq=0 out_seq=0 2024-09-16T02:04:59.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.547+0000 7f6f06000640 10 client.14904 ms_handle_connect on v2:172.21.15.136:6836/236571207 2024-09-16T02:04:59.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.547+0000 7f6f06000640 1 -- 172.21.15.103:0/2596220322 <== mds.1 v2:172.21.15.136:6836/236571207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5588021a7b30 con 0x5588022c5530 2024-09-16T02:04:59.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.547+0000 7f6f06000640 10 client.14904 handle_command_reply: tid=0 2024-09-16T02:04:59.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 client.14904 shutdown 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2596220322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 msgr2=0x55880223f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223f5e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6f140098b0 tx=0x7f6f14012e60 comp rx=0 tx=0).stop 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2596220322 shutdown_connections 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558802245510 0x558802240060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x5588022c5530 0x5588022c4240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55880223ac50 0x55880223fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 --2- 172.21.15.103:0/2596220322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55880223a330 0x55880223f5e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2596220322 >> 172.21.15.103:0/2596220322 conn(0x5588022368c0 msgr2=0x558802250cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f06000640 0 client.14904 ms_handle_reset on v2:172.21.15.136:6836/236571207 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2596220322 shutdown_connections 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 1 -- 172.21.15.103:0/2596220322 wait complete. 2024-09-16T02:04:59.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.548+0000 7f6f25dbd740 20 client.14904 trim_cache size 0 max 16384 2024-09-16T02:04:59.551 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:04:59.551 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "a20dd7a4-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:04:59.551 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-16T02:04:59.551 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24506, 2024-09-16T02:04:59.551 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-16T02:04:59.551 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-16T02:04:59.551 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-16T02:04:59.551 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 73.466555077999999, 2024-09-16T02:04:59.551 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 23, 2024-09-16T02:04:59.551 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 79, 2024-09-16T02:04:59.552 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 77, 2024-09-16T02:04:59.552 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 106.22551277399999 2024-09-16T02:04:59.552 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:04:59.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 -- 172.21.15.103:0/3434548594 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f000a3350 msgr2=0x7f6f000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f000a3350 0x7f6f000a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c098430 tx=0x7f6f0c0d2f90 comp rx=0 tx=0).stop 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 -- 172.21.15.103:0/3434548594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c101de0 msgr2=0x7f6f1c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c101de0 0x7f6f1c0744f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00d6d0 tx=0x7f6f0c00db90 comp rx=0 tx=0).stop 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 -- 172.21.15.103:0/3434548594 shutdown_connections 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6f000a3350 0x7f6f000a5810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f1c106f10 0x7f6f1c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c102790 0x7f6f1c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 --2- 172.21.15.103:0/3434548594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c101de0 0x7f6f1c0744f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 -- 172.21.15.103:0/3434548594 >> 172.21.15.103:0/3434548594 conn(0x7f6f1c0fd8c0 msgr2=0x7f6f1c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 -- 172.21.15.103:0/3434548594 shutdown_connections 2024-09-16T02:04:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:04:59.549+0000 7f6f23200640 1 -- 172.21.15.103:0/3434548594 wait complete. 2024-09-16T02:04:59.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:04:59 smithi103 ceph-mon[30398]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:04:59.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:04:59 smithi172 ceph-mon[35340]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:00.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:04:59 smithi136 ceph-mon[35323]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:00.403 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T02:05:00.773 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:01.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.345+0000 7fb8d6800640 1 -- 172.21.15.103:0/2737309636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8d0102790 msgr2=0x7fb8d01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:01.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.345+0000 7fb8d6800640 1 --2- 172.21.15.103:0/2737309636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8d0102790 0x7fb8d01067e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009900 tx=0x7fb8b802a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.346+0000 7fb8d6800640 1 -- 172.21.15.103:0/2737309636 shutdown_connections 2024-09-16T02:05:01.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.346+0000 7fb8d6800640 1 --2- 172.21.15.103:0/2737309636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8d0106f10 0x7fb8d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.346+0000 7fb8d6800640 1 --2- 172.21.15.103:0/2737309636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8d0102790 0x7fb8d01067e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.346+0000 7fb8d6800640 1 --2- 172.21.15.103:0/2737309636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0101de0 0x7fb8d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.346+0000 7fb8d6800640 1 -- 172.21.15.103:0/2737309636 >> 172.21.15.103:0/2737309636 conn(0x7fb8d00fd8c0 msgr2=0x7fb8d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:01.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.346+0000 7fb8d6800640 1 -- 172.21.15.103:0/2737309636 shutdown_connections 2024-09-16T02:05:01.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.346+0000 7fb8d6800640 1 -- 172.21.15.103:0/2737309636 wait complete. 2024-09-16T02:05:01.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8d6800640 1 Processor -- start 2024-09-16T02:05:01.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8d6800640 1 -- start start 2024-09-16T02:05:01.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8d6800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0101de0 0x7fb8d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8d6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8d0102790 0x7fb8d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8d6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8d0106f10 0x7fb8d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8d6800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0111fc0 con 0x7fb8d0106f10 2024-09-16T02:05:01.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8d6800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0111e40 con 0x7fb8d0102790 2024-09-16T02:05:01.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8d6800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0112140 con 0x7fb8d0101de0 2024-09-16T02:05:01.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8d0106f10 0x7fb8d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8cf400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0101de0 0x7fb8d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8d0106f10 0x7fb8d0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35326/0 (socket says 172.21.15.103:35326) 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8c6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8d0102790 0x7fb8d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.347+0000 7fb8cfe00640 1 -- 172.21.15.103:0/885851069 learned_addr learned my addr 172.21.15.103:0/885851069 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.348+0000 7fb8cf400640 1 -- 172.21.15.103:0/885851069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8d0102790 msgr2=0x7fb8d0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.348+0000 7fb8cf400640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8d0102790 0x7fb8d0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.348+0000 7fb8cf400640 1 -- 172.21.15.103:0/885851069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8d0106f10 msgr2=0x7fb8d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.348+0000 7fb8cf400640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8d0106f10 0x7fb8d0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.348+0000 7fb8cf400640 1 -- 172.21.15.103:0/885851069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d01a6920 con 0x7fb8d0101de0 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.348+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8d0106f10 0x7fb8d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.348+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8d0102790 0x7fb8d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.348+0000 7fb8cf400640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0101de0 0x7fb8d00744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0005fd0 tx=0x7fb8c000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.349+0000 7fb8ccc00640 1 -- 172.21.15.103:0/885851069 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0009c30 con 0x7fb8d0101de0 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.349+0000 7fb8d6800640 1 -- 172.21.15.103:0/885851069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d01a6c10 con 0x7fb8d0101de0 2024-09-16T02:05:01.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.349+0000 7fb8ccc00640 1 -- 172.21.15.103:0/885851069 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c0009dd0 con 0x7fb8d0101de0 2024-09-16T02:05:01.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.349+0000 7fb8d6800640 1 -- 172.21.15.103:0/885851069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01a7170 con 0x7fb8d0101de0 2024-09-16T02:05:01.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.349+0000 7fb8ccc00640 1 -- 172.21.15.103:0/885851069 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0020060 con 0x7fb8d0101de0 2024-09-16T02:05:01.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8d9443740 1 Processor -- start 2024-09-16T02:05:01.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8d9443740 1 -- start start 2024-09-16T02:05:01.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48529710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529c50 0x55bb48533fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb48536900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8d9443740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb4848f0b0 con 0x55bb48529330 2024-09-16T02:05:01.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8d9443740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bb48436ab0 con 0x55bb48534510 2024-09-16T02:05:01.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8d9443740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bb4848b070 con 0x55bb48529c50 2024-09-16T02:05:01.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8ccc00640 1 -- 172.21.15.103:0/885851069 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb8c00035f0 con 0x7fb8d0101de0 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48529710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48529710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35338/0 (socket says 172.21.15.103:35338) 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8cfe00640 1 -- 172.21.15.103:0/2846213963 learned_addr learned my addr 172.21.15.103:0/2846213963 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.352+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/2846213963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb48536900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.353+0000 7fb8cf400640 1 --2- 172.21.15.103:0/2846213963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529c50 0x55bb48533fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.353+0000 7fb8cfe00640 1 -- 172.21.15.103:0/2846213963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529c50 msgr2=0x55bb48533fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.353+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/2846213963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529c50 0x55bb48533fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.353+0000 7fb8cfe00640 1 -- 172.21.15.103:0/2846213963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 msgr2=0x55bb48536900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.353+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/2846213963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb48536900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.353+0000 7fb8cfe00640 1 -- 172.21.15.103:0/2846213963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb484434a0 con 0x55bb48529330 2024-09-16T02:05:01.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.353+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/2846213963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48529710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8011780 tx=0x7fb8c8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aeabbe26601a16c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:01.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.353+0000 7fb8c4c00640 1 -- 172.21.15.103:0/2846213963 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c800fb40 con 0x55bb48529330 2024-09-16T02:05:01.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.353+0000 7fb8c4c00640 1 -- 172.21.15.103:0/2846213963 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c8011e90 con 0x55bb48529330 2024-09-16T02:05:01.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.353+0000 7fb8c4c00640 1 -- 172.21.15.103:0/2846213963 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c801a650 con 0x55bb48529330 2024-09-16T02:05:01.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.354+0000 7fb8ccc00640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb8b40a32d0 0x7fb8b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.354+0000 7fb8ccc00640 1 -- 172.21.15.103:0/885851069 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fb8c00cf2e0 con 0x7fb8d0101de0 2024-09-16T02:05:01.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.354+0000 7fb8cf400640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb8b40a32d0 0x7fb8b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.354+0000 7fb8d9443740 1 -- 172.21.15.103:0/2846213963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 msgr2=0x55bb48529710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.354+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2846213963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48529710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8011780 tx=0x7fb8c8011c40 comp rx=0 tx=0).stop 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.354+0000 7fb8d9443740 1 -- 172.21.15.103:0/2846213963 shutdown_connections 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.354+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2846213963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb48536900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.354+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2846213963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529c50 0x55bb48533fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.354+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2846213963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48529710 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.354+0000 7fb8d9443740 1 -- 172.21.15.103:0/2846213963 >> 172.21.15.103:0/2846213963 conn(0x55bb485258c0 msgr2=0x55bb48525ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.355+0000 7fb8d9443740 1 -- 172.21.15.103:0/2846213963 shutdown_connections 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.355+0000 7fb8d9443740 1 -- 172.21.15.103:0/2846213963 wait complete. 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.355+0000 7fb8cf400640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb8b40a32d0 0x7fb8b40a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb8c009b770 tx=0x7fb8c0004610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.355+0000 7fb8d9443740 1 -- start start 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.355+0000 7fb8d9443740 1 Processor -- start 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48531510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48529c50 0x55bb48531a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48534510 0x55bb48531f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8d9443740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb48537060 con 0x55bb48529330 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8d9443740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bb48417920 con 0x55bb48529c50 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8d9443740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bb485371e0 con 0x55bb48534510 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8d9443740 10 client.0 _ll_register_callbacks cb 0x55bb4843b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48531510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48531510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35352/0 (socket says 172.21.15.103:35352) 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8cfe00640 1 -- 172.21.15.103:0/2608199525 learned_addr learned my addr 172.21.15.103:0/2608199525 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48529c50 0x55bb48531a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.356+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48534510 0x55bb48531f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8cfe00640 1 -- 172.21.15.103:0/2608199525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48534510 msgr2=0x55bb48531f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:01.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48534510 0x55bb48531f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8cfe00640 1 -- 172.21.15.103:0/2608199525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48529c50 msgr2=0x55bb48531a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48529c50 0x55bb48531a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8cfe00640 1 -- 172.21.15.103:0/2608199525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb482002b0 con 0x55bb48529330 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48531510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8012fd0 tx=0x7fb8c8012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821d55811f3e0b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8bd600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8bd600640 1 -- 172.21.15.103:0/2608199525 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c802f060 con 0x55bb48529330 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8bd600640 1 -- 172.21.15.103:0/2608199525 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fb8c8012450 con 0x55bb48529330 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8d9443740 1 -- 172.21.15.103:0/2608199525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8c8011e90 con 0x55bb48529330 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8bd600640 1 -- 172.21.15.103:0/2608199525 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb8c8012650 con 0x55bb48529330 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8d9443740 10 client.14916 fetch_fsmap learned FSMap version 23 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.357+0000 7fb8d9443740 1 -- 172.21.15.103:0/2608199525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55bb48542df0 con 0x55bb48529330 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.358+0000 7fb8bd600640 1 -- 172.21.15.103:0/2608199525 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c8011e90 con 0x55bb48529330 2024-09-16T02:05:01.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.358+0000 7fb8bd600640 1 -- 172.21.15.103:0/2608199525 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb8c8026040 con 0x55bb48529330 2024-09-16T02:05:01.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.358+0000 7fb8d9443740 10 client.14916 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:05:01.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.358+0000 7fb8d9443740 10 client.14916 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T02:05:01.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.358+0000 7fb8d9443740 20 client.14916 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:01.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.358+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55bb48544250 0x55bb48546430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.358+0000 7fb8d9443740 4 client.14916 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:01.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.358+0000 7fb8d9443740 1 -- 172.21.15.103:0/2608199525 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb4849c620 con 0x55bb48544250 2024-09-16T02:05:01.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.359+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55bb48544250 0x55bb48546430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.359+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55bb48544250 0x55bb48546430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eab2370d09c68f5 server_cookie=abf68481db139129 in_seq=0 out_seq=0 2024-09-16T02:05:01.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.359+0000 7fb8bd600640 10 client.14916 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:01.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8bd600640 1 -- 172.21.15.103:0/2608199525 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb4849c620 con 0x55bb48544250 2024-09-16T02:05:01.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8bd600640 10 client.14916 handle_command_reply: tid=0 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8d9443740 1 client.14916 shutdown 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8d9443740 1 -- 172.21.15.103:0/2608199525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 msgr2=0x55bb48531510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48531510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8012fd0 tx=0x7fb8c8012e60 comp rx=0 tx=0).stop 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8d9443740 1 -- 172.21.15.103:0/2608199525 shutdown_connections 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55bb48544250 0x55bb48546430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48534510 0x55bb48531f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48529c50 0x55bb48531a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2608199525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529330 0x55bb48531510 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8d9443740 1 -- 172.21.15.103:0/2608199525 >> 172.21.15.103:0/2608199525 conn(0x55bb485258c0 msgr2=0x55bb4852b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.360+0000 7fb8bd600640 0 client.14916 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.361+0000 7fb8d9443740 1 -- 172.21.15.103:0/2608199525 shutdown_connections 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.361+0000 7fb8d9443740 1 -- 172.21.15.103:0/2608199525 wait complete. 2024-09-16T02:05:01.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.361+0000 7fb8d9443740 20 client.14916 trim_cache size 0 max 16384 2024-09-16T02:05:01.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.379+0000 7fb8d9443740 1 Processor -- start 2024-09-16T02:05:01.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.379+0000 7fb8d9443740 1 -- start start 2024-09-16T02:05:01.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.379+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 0x55bb48530dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.379+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb4852e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.379+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb4852ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.379+0000 7fb8d9443740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb48544770 con 0x55bb48529c50 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.379+0000 7fb8d9443740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bb48544e40 con 0x55bb48534510 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.379+0000 7fb8d9443740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bb48545510 con 0x55bb48529330 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8c6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb4852e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8c6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb4852e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35364/0 (socket says 172.21.15.103:35364) 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8c6a00640 1 -- 172.21.15.103:0/2512001081 learned_addr learned my addr 172.21.15.103:0/2512001081 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8c6a00640 1 -- 172.21.15.103:0/2512001081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 msgr2=0x55bb48530dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/2512001081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 0x55bb48530dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/2512001081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 0x55bb48530dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8c6a00640 1 -- 172.21.15.103:0/2512001081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 msgr2=0x55bb4852ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/2512001081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb4852ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8c6a00640 1 -- 172.21.15.103:0/2512001081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb48526550 con 0x55bb48529c50 2024-09-16T02:05:01.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/2512001081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 0x55bb48530dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:05:01.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/2512001081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb4852e770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009900 tx=0x7fb8b8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcf46143ac72e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:01.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.380+0000 7fb8bfe00640 1 -- 172.21.15.103:0/2512001081 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8035040 con 0x55bb48529c50 2024-09-16T02:05:01.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8bfe00640 1 -- 172.21.15.103:0/2512001081 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b80055e0 con 0x55bb48529c50 2024-09-16T02:05:01.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8bfe00640 1 -- 172.21.15.103:0/2512001081 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8031dd0 con 0x55bb48529c50 2024-09-16T02:05:01.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8d9443740 1 -- 172.21.15.103:0/2512001081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 msgr2=0x55bb4852e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:01.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2512001081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb4852e770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009900 tx=0x7fb8b8005c00 comp rx=0 tx=0).stop 2024-09-16T02:05:01.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8d9443740 1 -- 172.21.15.103:0/2512001081 shutdown_connections 2024-09-16T02:05:01.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2512001081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb4852ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2512001081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb4852e770 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8d9443740 1 --2- 172.21.15.103:0/2512001081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 0x55bb48530dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8d9443740 1 -- 172.21.15.103:0/2512001081 >> 172.21.15.103:0/2512001081 conn(0x55bb485258c0 msgr2=0x55bb4852fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:01.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8d9443740 1 -- 172.21.15.103:0/2512001081 shutdown_connections 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.381+0000 7fb8d9443740 1 -- 172.21.15.103:0/2512001081 wait complete. 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.382+0000 7fb8d9443740 1 -- start start 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.382+0000 7fb8d9443740 1 Processor -- start 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.382+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 0x55bb485486d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.382+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb48548c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.382+0000 7fb8d9443740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb48549190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.382+0000 7fb8d9443740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb485462b0 con 0x55bb48529c50 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.382+0000 7fb8d9443740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bb48545be0 con 0x55bb48534510 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8d9443740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bb48546980 con 0x55bb48529330 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8d9443740 10 client.0 _ll_register_callbacks cb 0x7fb8c8004db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8c6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb48548c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8c6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb48548c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35378/0 (socket says 172.21.15.103:35378) 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8c6a00640 1 -- 172.21.15.103:0/82924716 learned_addr learned my addr 172.21.15.103:0/82924716 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 0x55bb485486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb48549190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8c6a00640 1 -- 172.21.15.103:0/82924716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 msgr2=0x55bb485486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 0x55bb485486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8c6a00640 1 -- 172.21.15.103:0/82924716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 msgr2=0x55bb48549190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb48549190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8c6a00640 1 -- 172.21.15.103:0/82924716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4854d1b0 con 0x55bb48529c50 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8c6a00640 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb48548c10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb8b802a910 tx=0x7fb8b802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da4ed287709addc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.383+0000 7fb8bd600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:05:01.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.384+0000 7fb8bd600640 1 -- 172.21.15.103:0/82924716 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8035040 con 0x55bb48529c50 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.384+0000 7fb8d9443740 1 -- 172.21.15.103:0/82924716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8b80055e0 con 0x55bb48529c50 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.384+0000 7fb8bd600640 1 -- 172.21.15.103:0/82924716 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fb8b8004760 con 0x55bb48529c50 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.384+0000 7fb8bd600640 1 -- 172.21.15.103:0/82924716 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb8b8031e60 con 0x55bb48529c50 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.384+0000 7fb8d9443740 10 client.14922 fetch_fsmap learned FSMap version 23 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.384+0000 7fb8d9443740 1 -- 172.21.15.103:0/82924716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55bb485afc30 con 0x55bb48529c50 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.384+0000 7fb8bd600640 1 -- 172.21.15.103:0/82924716 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b80055e0 con 0x55bb48529c50 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.384+0000 7fb8bd600640 1 -- 172.21.15.103:0/82924716 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb8b8003b30 con 0x55bb48529c50 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.384+0000 7fb8d9443740 10 client.14922 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.385+0000 7fb8d9443740 10 client.14922 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.385+0000 7fb8d9443740 20 client.14922 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.385+0000 7fb8d9443740 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55bb485b4260 0x55bb485b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.385+0000 7fb8d9443740 4 client.14922 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.385+0000 7fb8d9443740 1 -- 172.21.15.103:0/82924716 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "status"}) -- 0x55bb48496b30 con 0x55bb485b4260 2024-09-16T02:05:01.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.385+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55bb485b4260 0x55bb485b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:01.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8bd600640 10 client.14922 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:01.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8cfe00640 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55bb485b4260 0x55bb485b2f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3faa58a892d2376 server_cookie=9d2ed756e7be30ec in_seq=0 out_seq=0 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8bd600640 1 -- 172.21.15.103:0/82924716 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55bb48496b30 con 0x55bb485b4260 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8bd600640 10 client.14922 handle_command_reply: tid=0 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 client.14922 shutdown 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 -- 172.21.15.103:0/82924716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 msgr2=0x55bb48548c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb48548c10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb8b802a910 tx=0x7fb8b802a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 -- 172.21.15.103:0/82924716 shutdown_connections 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55bb485b4260 0x55bb485b2f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb48534510 0x55bb48549190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb48529c50 0x55bb48548c10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 --2- 172.21.15.103:0/82924716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb48529330 0x55bb485486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8bd600640 0 client.14922 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 -- 172.21.15.103:0/82924716 >> 172.21.15.103:0/82924716 conn(0x55bb485258c0 msgr2=0x55bb4852fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:01.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 -- 172.21.15.103:0/82924716 shutdown_connections 2024-09-16T02:05:01.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 1 -- 172.21.15.103:0/82924716 wait complete. 2024-09-16T02:05:01.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.386+0000 7fb8d9443740 20 client.14922 trim_cache size 0 max 16384 2024-09-16T02:05:01.392 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:05:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "a20dd7a4-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:05:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-16T02:05:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24545, 2024-09-16T02:05:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-16T02:05:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-16T02:05:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-16T02:05:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 83.354270073999999, 2024-09-16T02:05:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 23, 2024-09-16T02:05:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 79, 2024-09-16T02:05:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 77, 2024-09-16T02:05:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 96.144361863 2024-09-16T02:05:01.393 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:05:01.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 -- 172.21.15.103:0/885851069 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb8b40a32d0 msgr2=0x7fb8b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb8b40a32d0 0x7fb8b40a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb8c009b770 tx=0x7fb8c0004610 comp rx=0 tx=0).stop 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 -- 172.21.15.103:0/885851069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0101de0 msgr2=0x7fb8d00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0101de0 0x7fb8d00744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0005fd0 tx=0x7fb8c000dde0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 -- 172.21.15.103:0/885851069 shutdown_connections 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb8b40a32d0 0x7fb8b40a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8d0106f10 0x7fb8d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8d0102790 0x7fb8d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 --2- 172.21.15.103:0/885851069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0101de0 0x7fb8d00744f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 -- 172.21.15.103:0/885851069 >> 172.21.15.103:0/885851069 conn(0x7fb8d00fd8c0 msgr2=0x7fb8d00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 -- 172.21.15.103:0/885851069 shutdown_connections 2024-09-16T02:05:01.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:01.387+0000 7fb8d6800640 1 -- 172.21.15.103:0/885851069 wait complete. 2024-09-16T02:05:01.927 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:01 smithi103 ceph-mon[30398]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:05:02.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:01 smithi136 ceph-mon[35323]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:05:02.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:01 smithi172 ceph-mon[35340]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:05:02.084 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T02:05:02.085 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T02:05:02.096 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-16T02:05:02.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T02:05:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:02.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.979+0000 7fcfa1800640 1 -- 172.21.15.103:0/2442734447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf9c105920 msgr2=0x7fcf9c10c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.979+0000 7fcfa1800640 1 --2- 172.21.15.103:0/2442734447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf9c105920 0x7fcf9c10c240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009ca0 tx=0x7fcf9402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.979+0000 7fcfa1800640 1 -- 172.21.15.103:0/2442734447 shutdown_connections 2024-09-16T02:05:02.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.979+0000 7fcfa1800640 1 --2- 172.21.15.103:0/2442734447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf9c105920 0x7fcf9c10c240 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.979+0000 7fcfa1800640 1 --2- 172.21.15.103:0/2442734447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf9c104f60 0x7fcf9c1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.979+0000 7fcfa1800640 1 --2- 172.21.15.103:0/2442734447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c103e00 0x7fcf9c104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.979+0000 7fcfa1800640 1 -- 172.21.15.103:0/2442734447 >> 172.21.15.103:0/2442734447 conn(0x7fcf9c0ff5b0 msgr2=0x7fcf9c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:02.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.979+0000 7fcfa1800640 1 -- 172.21.15.103:0/2442734447 shutdown_connections 2024-09-16T02:05:02.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.979+0000 7fcfa1800640 1 -- 172.21.15.103:0/2442734447 wait complete. 2024-09-16T02:05:02.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.980+0000 7fcfa1800640 1 Processor -- start 2024-09-16T02:05:02.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.980+0000 7fcfa1800640 1 -- start start 2024-09-16T02:05:02.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.980+0000 7fcfa1800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c103e00 0x7fcf9c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.980+0000 7fcfa1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf9c104f60 0x7fcf9c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.980+0000 7fcfa1800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf9c105920 0x7fcf9c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.980+0000 7fcfa1800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c10fa90 con 0x7fcf9c105920 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.980+0000 7fcfa1800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c10f910 con 0x7fcf9c104f60 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.980+0000 7fcfa1800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c10fc10 con 0x7fcf9c103e00 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf9b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf9c105920 0x7fcf9c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf9b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf9c105920 0x7fcf9c1a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35402/0 (socket says 172.21.15.103:35402) 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf9b400640 1 -- 172.21.15.103:0/990915914 learned_addr learned my addr 172.21.15.103:0/990915914 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf93e00640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf9c104f60 0x7fcf9c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf9aa00640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c103e00 0x7fcf9c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf9b400640 1 -- 172.21.15.103:0/990915914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c103e00 msgr2=0x7fcf9c19a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf9b400640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c103e00 0x7fcf9c19a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf9b400640 1 -- 172.21.15.103:0/990915914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf9c104f60 msgr2=0x7fcf9c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:02.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf9b400640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf9c104f60 0x7fcf9c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:02.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf9b400640 1 -- 172.21.15.103:0/990915914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c1a1c80 con 0x7fcf9c105920 2024-09-16T02:05:02.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.981+0000 7fcf9b400640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf9c105920 0x7fcf9c1a1610 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcf94031ae0 tx=0x7fcf94031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:02.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.982+0000 7fcf93400640 1 -- 172.21.15.103:0/990915914 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94003c30 con 0x7fcf9c105920 2024-09-16T02:05:02.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.982+0000 7fcfa1800640 1 -- 172.21.15.103:0/990915914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c1a1f10 con 0x7fcf9c105920 2024-09-16T02:05:02.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.982+0000 7fcf93400640 1 -- 172.21.15.103:0/990915914 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf94003dd0 con 0x7fcf9c105920 2024-09-16T02:05:02.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.982+0000 7fcf93400640 1 -- 172.21.15.103:0/990915914 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9402fcb0 con 0x7fcf9c105920 2024-09-16T02:05:02.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.982+0000 7fcfa1800640 1 -- 172.21.15.103:0/990915914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c1a23c0 con 0x7fcf9c105920 2024-09-16T02:05:02.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.983+0000 7fcfa1800640 1 -- 172.21.15.103:0/990915914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9c10b190 con 0x7fcf9c105920 2024-09-16T02:05:02.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.985+0000 7fcf93400640 1 -- 172.21.15.103:0/990915914 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcf94004050 con 0x7fcf9c105920 2024-09-16T02:05:02.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.986+0000 7fcf93400640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf800a3300 0x7fcf800a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:02.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.986+0000 7fcf93400640 1 -- 172.21.15.103:0/990915914 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fcf940f0c00 con 0x7fcf9c105920 2024-09-16T02:05:02.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.986+0000 7fcf9aa00640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf800a3300 0x7fcf800a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:02.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.987+0000 7fcf9aa00640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf800a3300 0x7fcf800a57c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00bbe0 tx=0x7fcf8c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:02.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:02.990+0000 7fcf93400640 1 -- 172.21.15.103:0/990915914 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf940bd980 con 0x7fcf9c105920 2024-09-16T02:05:03.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.180+0000 7fcfa1800640 1 -- 172.21.15.103:0/990915914 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fcf9c062630 con 0x7fcf800a3300 2024-09-16T02:05:03.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.182+0000 7fcf93400640 1 -- 172.21.15.103:0/990915914 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fcf9c062630 con 0x7fcf800a3300 2024-09-16T02:05:03.183 INFO:teuthology.orchestra.run.smithi103.stdout:Backend: cephadm 2024-09-16T02:05:03.183 INFO:teuthology.orchestra.run.smithi103.stdout:Available: Yes 2024-09-16T02:05:03.183 INFO:teuthology.orchestra.run.smithi103.stdout:Paused: No 2024-09-16T02:05:03.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 -- 172.21.15.103:0/990915914 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf800a3300 msgr2=0x7fcf800a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:03.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf800a3300 0x7fcf800a57c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00bbe0 tx=0x7fcf8c00b390 comp rx=0 tx=0).stop 2024-09-16T02:05:03.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 -- 172.21.15.103:0/990915914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf9c105920 msgr2=0x7fcf9c1a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:03.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf9c105920 0x7fcf9c1a1610 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcf94031ae0 tx=0x7fcf94031ee0 comp rx=0 tx=0).stop 2024-09-16T02:05:03.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 -- 172.21.15.103:0/990915914 shutdown_connections 2024-09-16T02:05:03.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcf800a3300 0x7fcf800a57c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:03.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf9c105920 0x7fcf9c1a1610 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:03.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf9c104f60 0x7fcf9c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:03.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 --2- 172.21.15.103:0/990915914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c103e00 0x7fcf9c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:03.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 -- 172.21.15.103:0/990915914 >> 172.21.15.103:0/990915914 conn(0x7fcf9c0ff5b0 msgr2=0x7fcf9c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:03.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.184+0000 7fcfa1800640 1 -- 172.21.15.103:0/990915914 shutdown_connections 2024-09-16T02:05:03.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:03.185+0000 7fcfa1800640 1 -- 172.21.15.103:0/990915914 wait complete. 2024-09-16T02:05:03.953 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:03 smithi103 ceph-mon[30398]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:03.990 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T02:05:04.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:03 smithi136 ceph-mon[35323]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:04.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:03 smithi172 ceph-mon[35340]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:04.739 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:04 smithi103 ceph-mon[30398]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.844+0000 7fe114a00640 1 -- 172.21.15.103:0/1370996448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe110104e50 msgr2=0x7fe1101052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.844+0000 7fe114a00640 1 --2- 172.21.15.103:0/1370996448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe110104e50 0x7fe1101052d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009900 tx=0x7fe0f802a410 comp rx=0 tx=0).stop 2024-09-16T02:05:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.844+0000 7fe114a00640 1 -- 172.21.15.103:0/1370996448 shutdown_connections 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.844+0000 7fe114a00640 1 --2- 172.21.15.103:0/1370996448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110105810 0x7fe11010c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.844+0000 7fe114a00640 1 --2- 172.21.15.103:0/1370996448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe110104e50 0x7fe1101052d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.844+0000 7fe114a00640 1 --2- 172.21.15.103:0/1370996448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110103c50 0x7fe110104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.844+0000 7fe114a00640 1 -- 172.21.15.103:0/1370996448 >> 172.21.15.103:0/1370996448 conn(0x7fe1100ff3e0 msgr2=0x7fe110101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.844+0000 7fe114a00640 1 -- 172.21.15.103:0/1370996448 shutdown_connections 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.844+0000 7fe114a00640 1 -- 172.21.15.103:0/1370996448 wait complete. 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.845+0000 7fe114a00640 1 Processor -- start 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.845+0000 7fe114a00640 1 -- start start 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.845+0000 7fe114a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110103c50 0x7fe11019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.845+0000 7fe114a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe110104e50 0x7fe11019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.845+0000 7fe114a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110105810 0x7fe1101a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:04.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.845+0000 7fe114a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe11010f9d0 con 0x7fe110104e50 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.845+0000 7fe114a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe11010f850 con 0x7fe110105810 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.845+0000 7fe114a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe11010fb50 con 0x7fe110103c50 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.845+0000 7fe105e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe110104e50 0x7fe11019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.846+0000 7fe105e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe110104e50 0x7fe11019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35414/0 (socket says 172.21.15.103:35414) 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.846+0000 7fe105e00640 1 -- 172.21.15.103:0/1138271279 learned_addr learned my addr 172.21.15.103:0/1138271279 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.846+0000 7fe10f200640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110105810 0x7fe1101a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.846+0000 7fe105e00640 1 -- 172.21.15.103:0/1138271279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110103c50 msgr2=0x7fe11019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.846+0000 7fe105e00640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110103c50 0x7fe11019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.846+0000 7fe105e00640 1 -- 172.21.15.103:0/1138271279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110105810 msgr2=0x7fe1101a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.846+0000 7fe105e00640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110105810 0x7fe1101a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.846+0000 7fe105e00640 1 -- 172.21.15.103:0/1138271279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1101a1cc0 con 0x7fe110104e50 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.846+0000 7fe105e00640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe110104e50 0x7fe11019a5d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009230 tx=0x7fe0f8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.847+0000 7fe107e00640 1 -- 172.21.15.103:0/1138271279 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f803f030 con 0x7fe110104e50 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.847+0000 7fe107e00640 1 -- 172.21.15.103:0/1138271279 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0f8003ce0 con 0x7fe110104e50 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.847+0000 7fe114a00640 1 -- 172.21.15.103:0/1138271279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1101a1f50 con 0x7fe110104e50 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.847+0000 7fe107e00640 1 -- 172.21.15.103:0/1138271279 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8035070 con 0x7fe110104e50 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.847+0000 7fe114a00640 1 -- 172.21.15.103:0/1138271279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1101a2420 con 0x7fe110104e50 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.848+0000 7fe114a00640 1 -- 172.21.15.103:0/1138271279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe11010b0f0 con 0x7fe110104e50 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.850+0000 7fe107e00640 1 -- 172.21.15.103:0/1138271279 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe0f8042030 con 0x7fe110104e50 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.851+0000 7fe107e00640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0f40a3300 0x7fe0f40a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.851+0000 7fe107e00640 1 -- 172.21.15.103:0/1138271279 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fe0f80f02c0 con 0x7fe110104e50 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.851+0000 7fe10e800640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0f40a3300 0x7fe0f40a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.852+0000 7fe10e800640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0f40a3300 0x7fe0f40a57c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe110104cb0 tx=0x7fe100006c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:04.855+0000 7fe107e00640 1 -- 172.21.15.103:0/1138271279 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0f80bce10 con 0x7fe110104e50 2024-09-16T02:05:05.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:04 smithi136 ceph-mon[35323]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:05.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.037+0000 7fe114a00640 1 -- 172.21.15.103:0/1138271279 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe110062630 con 0x7fe0f40a3300 2024-09-16T02:05:05.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:04 smithi172 ceph-mon[35340]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.051+0000 7fe107e00640 1 -- 172.21.15.103:0/1138271279 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fe110062630 con 0x7fe0f40a3300 2024-09-16T02:05:05.052 INFO:teuthology.orchestra.run.smithi103.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T02:05:05.052 INFO:teuthology.orchestra.run.smithi103.stdout:mds.a smithi103 running (102s) 19s ago 102s 12.7M - 19.1.1-326-g8921c8ef f1455bd0219c ddd596b8db3c 2024-09-16T02:05:05.052 INFO:teuthology.orchestra.run.smithi103.stdout:mds.b smithi136 running (100s) 33s ago 99s 25.6M - 19.1.1-326-g8921c8ef f1455bd0219c a47b2ae2e912 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mds.c smithi172 running (114s) 33s ago 114s 19.4M - 19.1.1-326-g8921c8ef f1455bd0219c 1a8f64a9b353 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mds.d smithi103 running (2m) 19s ago 2m 12.7M - 19.1.1-326-g8921c8ef f1455bd0219c 3a94b3dabb4f 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mds.e smithi136 running (119s) 33s ago 119s 16.1M - 19.1.1-326-g8921c8ef f1455bd0219c f79535200b0a 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mds.f smithi172 running (117s) 33s ago 116s 18.3M - 19.1.1-326-g8921c8ef f1455bd0219c 54d1f2756558 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mds.g smithi103 running (106s) 19s ago 105s 12.7M - 19.1.1-326-g8921c8ef f1455bd0219c e4778ffffc52 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mds.h smithi136 running (112s) 33s ago 111s 19.6M - 19.1.1-326-g8921c8ef f1455bd0219c 5bd2a4f82897 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mds.i smithi172 running (109s) 33s ago 109s 19.6M - 19.1.1-326-g8921c8ef f1455bd0219c bb31bc2221b4 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mds.j smithi103 running (2m) 19s ago 2m 12.7M - 19.1.1-326-g8921c8ef f1455bd0219c c975b03dbfa1 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mds.k smithi136 running (2m) 33s ago 2m 14.5M - 19.1.1-326-g8921c8ef f1455bd0219c 3c88d7cf0f4b 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mds.l smithi172 running (2m) 33s ago 2m 13.5M - 19.1.1-326-g8921c8ef f1455bd0219c 1f7e1640479f 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.x smithi103 *:9283,8765 running (9m) 19s ago 9m 508M - 19.1.1-326-g8921c8ef f1455bd0219c 9e0b235d2a04 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.y smithi136 *:8443,8765 running (7m) 33s ago 7m 443M - 19.1.1-326-g8921c8ef f1455bd0219c 1060f40eb505 2024-09-16T02:05:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.z smithi172 *:8443,8765 running (7m) 33s ago 7m 441M - 19.1.1-326-g8921c8ef f1455bd0219c bb33a43d08a8 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:mon.a smithi103 running (9m) 19s ago 9m 48.1M 2048M 19.1.1-326-g8921c8ef f1455bd0219c e60b64d004a0 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:mon.b smithi136 running (7m) 33s ago 7m 40.2M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 66e9a9454b02 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:mon.c smithi172 running (8m) 33s ago 8m 40.4M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 9e5b8f07d99b 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 smithi103 running (7m) 19s ago 7m 56.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c cab308f332a9 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 smithi136 running (6m) 33s ago 6m 57.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c b89abf5a26da 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 smithi172 running (6m) 33s ago 6m 57.8M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 5feed3346b0a 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 smithi103 running (5m) 19s ago 5m 45.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 551a78cea4e4 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 smithi136 running (5m) 33s ago 5m 45.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 6d0510a4df13 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 smithi172 running (5m) 33s ago 5m 46.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c f8994531df5e 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 smithi103 running (4m) 19s ago 4m 43.2M 4272M 19.1.1-326-g8921c8ef f1455bd0219c bad4a2358801 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 smithi136 running (4m) 33s ago 4m 53.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 0871134013f0 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 smithi172 running (3m) 33s ago 3m 56.4M 4272M 19.1.1-326-g8921c8ef f1455bd0219c f315adbd1f6c 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 smithi103 running (3m) 19s ago 3m 50.2M 4272M 19.1.1-326-g8921c8ef f1455bd0219c db171ea63b39 2024-09-16T02:05:05.054 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 smithi136 running (2m) 33s ago 2m 47.4M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 4eaa1abfdbb5 2024-09-16T02:05:05.055 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 smithi172 running (2m) 33s ago 2m 43.1M 4272M 19.1.1-326-g8921c8ef f1455bd0219c b4b6ade518dd 2024-09-16T02:05:05.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 -- 172.21.15.103:0/1138271279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0f40a3300 msgr2=0x7fe0f40a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:05.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0f40a3300 0x7fe0f40a57c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe110104cb0 tx=0x7fe100006c40 comp rx=0 tx=0).stop 2024-09-16T02:05:05.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 -- 172.21.15.103:0/1138271279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe110104e50 msgr2=0x7fe11019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:05.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe110104e50 0x7fe11019a5d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009230 tx=0x7fe0f8004430 comp rx=0 tx=0).stop 2024-09-16T02:05:05.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 -- 172.21.15.103:0/1138271279 shutdown_connections 2024-09-16T02:05:05.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe0f40a3300 0x7fe0f40a57c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:05.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110105810 0x7fe1101a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:05.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe110104e50 0x7fe11019a5d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:05.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 --2- 172.21.15.103:0/1138271279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110103c50 0x7fe11019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:05.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 -- 172.21.15.103:0/1138271279 >> 172.21.15.103:0/1138271279 conn(0x7fe1100ff3e0 msgr2=0x7fe1101082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:05.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 -- 172.21.15.103:0/1138271279 shutdown_connections 2024-09-16T02:05:05.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:05.054+0000 7fe114a00640 1 -- 172.21.15.103:0/1138271279 wait complete. 2024-09-16T02:05:05.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:05 smithi103 ceph-mon[30398]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:05.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T02:05:06.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:05 smithi136 ceph-mon[35323]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:06.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:05 smithi172 ceph-mon[35340]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:06.214 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:06.623 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:06 smithi103 ceph-mon[30398]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.733+0000 7f747ec00640 1 -- 172.21.15.103:0/2603584217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7478103600 msgr2=0x7f7478109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.733+0000 7f747ec00640 1 --2- 172.21.15.103:0/2603584217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7478103600 0x7f7478109e90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7470009900 tx=0x7f747002a410 comp rx=0 tx=0).stop 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.733+0000 7f747ec00640 1 -- 172.21.15.103:0/2603584217 shutdown_connections 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.733+0000 7f747ec00640 1 --2- 172.21.15.103:0/2603584217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7478103600 0x7f7478109e90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.733+0000 7f747ec00640 1 --2- 172.21.15.103:0/2603584217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7478102c40 0x7f74781030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.733+0000 7f747ec00640 1 --2- 172.21.15.103:0/2603584217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7478101a40 0x7f7478101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.733+0000 7f747ec00640 1 -- 172.21.15.103:0/2603584217 >> 172.21.15.103:0/2603584217 conn(0x7f74780fd1f0 msgr2=0x7f74780ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.733+0000 7f747ec00640 1 -- 172.21.15.103:0/2603584217 shutdown_connections 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.733+0000 7f747ec00640 1 -- 172.21.15.103:0/2603584217 wait complete. 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.734+0000 7f747ec00640 1 Processor -- start 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.734+0000 7f747ec00640 1 -- start start 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.734+0000 7f747ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7478101a40 0x7f7478199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.734+0000 7f747ec00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7478102c40 0x7f747819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.734+0000 7f747ec00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7478103600 0x7f74781a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.734+0000 7f747ec00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f747810d790 con 0x7f7478102c40 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.734+0000 7f747ec00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f747810d610 con 0x7f7478103600 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.734+0000 7f747ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f747810d910 con 0x7f7478101a40 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7478102c40 0x7f747819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747e200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7478103600 0x7f74781a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747e200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7478103600 0x7f74781a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:44600/0 (socket says 172.21.15.103:44600) 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747e200640 1 -- 172.21.15.103:0/1844583350 learned_addr learned my addr 172.21.15.103:0/1844583350 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747ce00640 1 -- 172.21.15.103:0/1844583350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7478101a40 msgr2=0x7f7478199f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747ce00640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7478101a40 0x7f7478199f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747ce00640 1 -- 172.21.15.103:0/1844583350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7478103600 msgr2=0x7f74781a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747ce00640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7478103600 0x7f74781a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747ce00640 1 -- 172.21.15.103:0/1844583350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74781a1b80 con 0x7f7478102c40 2024-09-16T02:05:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747e200640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7478103600 0x7f74781a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f747ce00640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7478102c40 0x7f747819a490 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f746400d6d0 tx=0x7f746400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.735+0000 7f7476000640 1 -- 172.21.15.103:0/1844583350 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464017070 con 0x7f7478102c40 2024-09-16T02:05:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.736+0000 7f7476000640 1 -- 172.21.15.103:0/1844583350 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7464009820 con 0x7f7478102c40 2024-09-16T02:05:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.736+0000 7f747ec00640 1 -- 172.21.15.103:0/1844583350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74781a1e70 con 0x7f7478102c40 2024-09-16T02:05:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.736+0000 7f7476000640 1 -- 172.21.15.103:0/1844583350 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464004d00 con 0x7f7478102c40 2024-09-16T02:05:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.736+0000 7f747ec00640 1 -- 172.21.15.103:0/1844583350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74781a23a0 con 0x7f7478102c40 2024-09-16T02:05:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.737+0000 7f747ec00640 1 -- 172.21.15.103:0/1844583350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74781a25d0 con 0x7f7478102c40 2024-09-16T02:05:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.739+0000 7f7476000640 1 -- 172.21.15.103:0/1844583350 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f74640099d0 con 0x7f7478102c40 2024-09-16T02:05:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.740+0000 7f7476000640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f74600a3300 0x7f74600a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.740+0000 7f7476000640 1 -- 172.21.15.103:0/1844583350 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f74640cf640 con 0x7f7478102c40 2024-09-16T02:05:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.740+0000 7f747d800640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f74600a3300 0x7f74600a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.741+0000 7f747d800640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f74600a3300 0x7f74600a57c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7478102aa0 tx=0x7f746c009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.744+0000 7f7476000640 1 -- 172.21.15.103:0/1844583350 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f746409c190 con 0x7f7478102c40 2024-09-16T02:05:06.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.928+0000 7f747ec00640 1 -- 172.21.15.103:0/1844583350 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7478075b40 con 0x7f74600a3300 2024-09-16T02:05:06.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.934+0000 7f7476000640 1 -- 172.21.15.103:0/1844583350 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7478075b40 con 0x7f74600a3300 2024-09-16T02:05:06.936 INFO:teuthology.orchestra.run.smithi103.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T02:05:06.936 INFO:teuthology.orchestra.run.smithi103.stdout:mds.all 12/12 34s ago 2m smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-09-16T02:05:06.937 INFO:teuthology.orchestra.run.smithi103.stdout:mgr 3/3 34s ago 7m smithi103=x;smithi136=y;smithi172=z;count:3 2024-09-16T02:05:06.937 INFO:teuthology.orchestra.run.smithi103.stdout:mon 3/3 34s ago 8m smithi103:172.21.15.103=a;smithi136:172.21.15.136=b;smithi172:172.21.15.172=c;count:3 2024-09-16T02:05:06.937 INFO:teuthology.orchestra.run.smithi103.stdout:osd 12 34s ago - 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.936+0000 7f747ec00640 1 -- 172.21.15.103:0/1844583350 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f74600a3300 msgr2=0x7f74600a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.936+0000 7f747ec00640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f74600a3300 0x7f74600a57c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7478102aa0 tx=0x7f746c009250 comp rx=0 tx=0).stop 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.936+0000 7f747ec00640 1 -- 172.21.15.103:0/1844583350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7478102c40 msgr2=0x7f747819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.936+0000 7f747ec00640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7478102c40 0x7f747819a490 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f746400d6d0 tx=0x7f746400db90 comp rx=0 tx=0).stop 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.937+0000 7f747ec00640 1 -- 172.21.15.103:0/1844583350 shutdown_connections 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.937+0000 7f747ec00640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f74600a3300 0x7f74600a57c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.937+0000 7f747ec00640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7478103600 0x7f74781a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.937+0000 7f747ec00640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7478102c40 0x7f747819a490 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.937+0000 7f747ec00640 1 --2- 172.21.15.103:0/1844583350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7478101a40 0x7f7478199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.937+0000 7f747ec00640 1 -- 172.21.15.103:0/1844583350 >> 172.21.15.103:0/1844583350 conn(0x7f74780fd1f0 msgr2=0x7f7478074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.937+0000 7f747ec00640 1 -- 172.21.15.103:0/1844583350 shutdown_connections 2024-09-16T02:05:06.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:06.937+0000 7f747ec00640 1 -- 172.21.15.103:0/1844583350 wait complete. 2024-09-16T02:05:07.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:06 smithi136 ceph-mon[35323]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:07.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:06 smithi172 ceph-mon[35340]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:07.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T02:05:07.949 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:07 smithi172 ceph-mon[35340]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:08.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:07 smithi103 ceph-mon[30398]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:08.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:07 smithi136 ceph-mon[35323]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:08.103 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.609+0000 7f2ad6000640 1 -- 172.21.15.103:0/2639581583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0104e50 msgr2=0x7f2ad01052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.609+0000 7f2ad6000640 1 --2- 172.21.15.103:0/2639581583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0104e50 0x7f2ad01052d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009900 tx=0x7f2abc02a410 comp rx=0 tx=0).stop 2024-09-16T02:05:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.609+0000 7f2ad6000640 1 -- 172.21.15.103:0/2639581583 shutdown_connections 2024-09-16T02:05:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.609+0000 7f2ad6000640 1 --2- 172.21.15.103:0/2639581583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ad0105810 0x7f2ad010c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.609+0000 7f2ad6000640 1 --2- 172.21.15.103:0/2639581583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0104e50 0x7f2ad01052d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.609+0000 7f2ad6000640 1 --2- 172.21.15.103:0/2639581583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ad0103c50 0x7f2ad0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.609+0000 7f2ad6000640 1 -- 172.21.15.103:0/2639581583 >> 172.21.15.103:0/2639581583 conn(0x7f2ad00ff3e0 msgr2=0x7f2ad0101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:08.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.610+0000 7f2ad6000640 1 -- 172.21.15.103:0/2639581583 shutdown_connections 2024-09-16T02:05:08.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.610+0000 7f2ad6000640 1 -- 172.21.15.103:0/2639581583 wait complete. 2024-09-16T02:05:08.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.610+0000 7f2ad6000640 1 Processor -- start 2024-09-16T02:05:08.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad6000640 1 -- start start 2024-09-16T02:05:08.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0103c50 0x7f2ad0199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad6000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ad0104e50 0x7f2ad019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ad0105810 0x7f2ad01a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad6000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010f920 con 0x7f2ad0103c50 2024-09-16T02:05:08.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad6000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010f7a0 con 0x7f2ad0105810 2024-09-16T02:05:08.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad6000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010faa0 con 0x7f2ad0104e50 2024-09-16T02:05:08.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad4c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0103c50 0x7f2ad0199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad4c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0103c50 0x7f2ad0199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49252/0 (socket says 172.21.15.103:49252) 2024-09-16T02:05:08.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad4c00640 1 -- 172.21.15.103:0/4137637535 learned_addr learned my addr 172.21.15.103:0/4137637535 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:08.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2acfe00640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ad0104e50 0x7f2ad019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.611+0000 7f2ad5600640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ad0105810 0x7f2ad01a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.612+0000 7f2ad4c00640 1 -- 172.21.15.103:0/4137637535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ad0104e50 msgr2=0x7f2ad019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.612+0000 7f2ad4c00640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ad0104e50 0x7f2ad019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.612+0000 7f2ad4c00640 1 -- 172.21.15.103:0/4137637535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ad0105810 msgr2=0x7f2ad01a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.612+0000 7f2ad4c00640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ad0105810 0x7f2ad01a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.612+0000 7f2ad4c00640 1 -- 172.21.15.103:0/4137637535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad01a1b70 con 0x7f2ad0103c50 2024-09-16T02:05:08.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.612+0000 7f2ad4c00640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0103c50 0x7f2ad0199f40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400e9c0 tx=0x7f2ac400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:08.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.612+0000 7f2acd600640 1 -- 172.21.15.103:0/4137637535 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400cd80 con 0x7f2ad0103c50 2024-09-16T02:05:08.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.612+0000 7f2acd600640 1 -- 172.21.15.103:0/4137637535 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ac40035f0 con 0x7f2ad0103c50 2024-09-16T02:05:08.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.612+0000 7f2acd600640 1 -- 172.21.15.103:0/4137637535 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4010690 con 0x7f2ad0103c50 2024-09-16T02:05:08.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.612+0000 7f2ad6000640 1 -- 172.21.15.103:0/4137637535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad01a1e60 con 0x7f2ad0103c50 2024-09-16T02:05:08.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.613+0000 7f2ad6000640 1 -- 172.21.15.103:0/4137637535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad01a2260 con 0x7f2ad0103c50 2024-09-16T02:05:08.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.614+0000 7f2ad6000640 1 -- 172.21.15.103:0/4137637535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad010b020 con 0x7f2ad0103c50 2024-09-16T02:05:08.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.615+0000 7f2acd600640 1 -- 172.21.15.103:0/4137637535 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2ac40037a0 con 0x7f2ad0103c50 2024-09-16T02:05:08.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.617+0000 7f2acd600640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ab80a3300 0x7f2ab80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:08.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.617+0000 7f2acd600640 1 -- 172.21.15.103:0/4137637535 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f2ac40cfeb0 con 0x7f2ad0103c50 2024-09-16T02:05:08.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.617+0000 7f2acfe00640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ab80a3300 0x7f2ab80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:08.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.618+0000 7f2acfe00640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ab80a3300 0x7f2ab80a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2abc031440 tx=0x7f2abc0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:08.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.620+0000 7f2acd600640 1 -- 172.21.15.103:0/4137637535 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ac409ca00 con 0x7f2ad0103c50 2024-09-16T02:05:08.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:08 smithi103 ceph-mon[30398]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.801+0000 7f2ad6000640 1 -- 172.21.15.103:0/4137637535 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2ad0062630 con 0x7f2ab80a3300 2024-09-16T02:05:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.803+0000 7f2acd600640 1 -- 172.21.15.103:0/4137637535 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f2ad0062630 con 0x7f2ab80a3300 2024-09-16T02:05:08.805 INFO:teuthology.orchestra.run.smithi103.stdout:HOST ADDR LABELS STATUS 2024-09-16T02:05:08.805 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 172.21.15.103 2024-09-16T02:05:08.805 INFO:teuthology.orchestra.run.smithi103.stdout:smithi136 172.21.15.136 2024-09-16T02:05:08.806 INFO:teuthology.orchestra.run.smithi103.stdout:smithi172 172.21.15.172 2024-09-16T02:05:08.806 INFO:teuthology.orchestra.run.smithi103.stdout:3 hosts in cluster 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.805+0000 7f2ad6000640 1 -- 172.21.15.103:0/4137637535 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ab80a3300 msgr2=0x7f2ab80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.805+0000 7f2ad6000640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ab80a3300 0x7f2ab80a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2abc031440 tx=0x7f2abc0315b0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.805+0000 7f2ad6000640 1 -- 172.21.15.103:0/4137637535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0103c50 msgr2=0x7f2ad0199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.805+0000 7f2ad6000640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0103c50 0x7f2ad0199f40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400e9c0 tx=0x7f2ac400ee80 comp rx=0 tx=0).stop 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.806+0000 7f2ad6000640 1 -- 172.21.15.103:0/4137637535 shutdown_connections 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.806+0000 7f2ad6000640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2ab80a3300 0x7f2ab80a57c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.806+0000 7f2ad6000640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ad0105810 0x7f2ad01a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.806+0000 7f2ad6000640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ad0104e50 0x7f2ad019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.806+0000 7f2ad6000640 1 --2- 172.21.15.103:0/4137637535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0103c50 0x7f2ad0199f40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.806+0000 7f2ad6000640 1 -- 172.21.15.103:0/4137637535 >> 172.21.15.103:0/4137637535 conn(0x7f2ad00ff3e0 msgr2=0x7f2ad01081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:08.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.806+0000 7f2ad6000640 1 -- 172.21.15.103:0/4137637535 shutdown_connections 2024-09-16T02:05:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:08.806+0000 7f2ad6000640 1 -- 172.21.15.103:0/4137637535 wait complete. 2024-09-16T02:05:09.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:08 smithi136 ceph-mon[35323]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:09.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:08 smithi172 ceph-mon[35340]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:09.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T02:05:09.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:09 smithi103 ceph-mon[30398]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:09.925 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:09 smithi136 ceph-mon[35323]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:10.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:09 smithi172 ceph-mon[35340]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.432+0000 7f49a8c00640 1 -- 172.21.15.103:0/3103263214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a4105810 msgr2=0x7f49a410c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.432+0000 7f49a8c00640 1 --2- 172.21.15.103:0/3103263214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a4105810 0x7f49a410c0a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f499c009ca0 tx=0x7f499c02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.432+0000 7f49a8c00640 1 -- 172.21.15.103:0/3103263214 shutdown_connections 2024-09-16T02:05:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.432+0000 7f49a8c00640 1 --2- 172.21.15.103:0/3103263214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a4105810 0x7f49a410c0a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.432+0000 7f49a8c00640 1 --2- 172.21.15.103:0/3103263214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49a4104e50 0x7f49a41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.432+0000 7f49a8c00640 1 --2- 172.21.15.103:0/3103263214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49a4103c50 0x7f49a4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.432+0000 7f49a8c00640 1 -- 172.21.15.103:0/3103263214 >> 172.21.15.103:0/3103263214 conn(0x7f49a40ff3e0 msgr2=0x7f49a4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.432+0000 7f49a8c00640 1 -- 172.21.15.103:0/3103263214 shutdown_connections 2024-09-16T02:05:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.432+0000 7f49a8c00640 1 -- 172.21.15.103:0/3103263214 wait complete. 2024-09-16T02:05:10.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.433+0000 7f49a8c00640 1 Processor -- start 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.433+0000 7f49a8c00640 1 -- start start 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a8c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a4103c50 0x7f49a4199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a8c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49a4104e50 0x7f49a419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a8c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49a4105810 0x7f49a41a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a8c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f49a410f9a0 con 0x7f49a4103c50 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a8c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49a410f820 con 0x7f49a4105810 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a8c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49a410fb20 con 0x7f49a4104e50 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a4103c50 0x7f49a4199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a4103c50 0x7f49a4199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49270/0 (socket says 172.21.15.103:49270) 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a3400640 1 -- 172.21.15.103:0/4211888670 learned_addr learned my addr 172.21.15.103:0/4211888670 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a3e00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49a4105810 0x7f49a41a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a2a00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49a4104e50 0x7f49a419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a3400640 1 -- 172.21.15.103:0/4211888670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49a4104e50 msgr2=0x7f49a419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a3400640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49a4104e50 0x7f49a419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.434+0000 7f49a3400640 1 -- 172.21.15.103:0/4211888670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49a4105810 msgr2=0x7f49a41a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.435+0000 7f49a3400640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49a4105810 0x7f49a41a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.435+0000 7f49a3400640 1 -- 172.21.15.103:0/4211888670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a41a1b40 con 0x7f49a4103c50 2024-09-16T02:05:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.435+0000 7f49a3400640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a4103c50 0x7f49a4199f10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f499400b450 tx=0x7f499400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.435+0000 7f499be00640 1 -- 172.21.15.103:0/4211888670 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994015020 con 0x7f49a4103c50 2024-09-16T02:05:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.435+0000 7f499be00640 1 -- 172.21.15.103:0/4211888670 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4994011d70 con 0x7f49a4103c50 2024-09-16T02:05:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.435+0000 7f49a8c00640 1 -- 172.21.15.103:0/4211888670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a41a1e30 con 0x7f49a4103c50 2024-09-16T02:05:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.435+0000 7f499be00640 1 -- 172.21.15.103:0/4211888670 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499400f340 con 0x7f49a4103c50 2024-09-16T02:05:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.435+0000 7f49a8c00640 1 -- 172.21.15.103:0/4211888670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a41a22b0 con 0x7f49a4103c50 2024-09-16T02:05:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.436+0000 7f49a8c00640 1 -- 172.21.15.103:0/4211888670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a410af80 con 0x7f49a4103c50 2024-09-16T02:05:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.438+0000 7f499be00640 1 -- 172.21.15.103:0/4211888670 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4994018020 con 0x7f49a4103c50 2024-09-16T02:05:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.439+0000 7f499be00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49880a3300 0x7f49880a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.440+0000 7f499be00640 1 -- 172.21.15.103:0/4211888670 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f49940cf400 con 0x7f49a4103c50 2024-09-16T02:05:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.440+0000 7f49a2a00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49880a3300 0x7f49880a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.440+0000 7f49a2a00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49880a3300 0x7f49880a57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f49a419b430 tx=0x7f4990005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.444+0000 7f499be00640 1 -- 172.21.15.103:0/4211888670 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f499401f020 con 0x7f49a4103c50 2024-09-16T02:05:10.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.631+0000 7f49a8c00640 1 -- 172.21.15.103:0/4211888670 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f49a4062630 con 0x7f49880a3300 2024-09-16T02:05:10.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.633+0000 7f499be00640 1 -- 172.21.15.103:0/4211888670 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f49a4062630 con 0x7f49880a3300 2024-09-16T02:05:10.635 INFO:teuthology.orchestra.run.smithi103.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T02:05:10.635 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T02:05:10.635 INFO:teuthology.orchestra.run.smithi103.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 2m ago Has a FileSystem, LVM detected 2024-09-16T02:05:10.635 INFO:teuthology.orchestra.run.smithi103.stdout:smithi172 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005M400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T02:05:10.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.635+0000 7f49a8c00640 1 -- 172.21.15.103:0/4211888670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49880a3300 msgr2=0x7f49880a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.635+0000 7f49a8c00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49880a3300 0x7f49880a57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f49a419b430 tx=0x7f4990005f90 comp rx=0 tx=0).stop 2024-09-16T02:05:10.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.635+0000 7f49a8c00640 1 -- 172.21.15.103:0/4211888670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a4103c50 msgr2=0x7f49a4199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:10.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.635+0000 7f49a8c00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a4103c50 0x7f49a4199f10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f499400b450 tx=0x7f499400b910 comp rx=0 tx=0).stop 2024-09-16T02:05:10.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.636+0000 7f49a8c00640 1 -- 172.21.15.103:0/4211888670 shutdown_connections 2024-09-16T02:05:10.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.636+0000 7f49a8c00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49880a3300 0x7f49880a57c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.636+0000 7f49a8c00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49a4105810 0x7f49a41a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.636+0000 7f49a8c00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49a4104e50 0x7f49a419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.636+0000 7f49a8c00640 1 --2- 172.21.15.103:0/4211888670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a4103c50 0x7f49a4199f10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:10.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.636+0000 7f49a8c00640 1 -- 172.21.15.103:0/4211888670 >> 172.21.15.103:0/4211888670 conn(0x7f49a40ff3e0 msgr2=0x7f49a4108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:10.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.636+0000 7f49a8c00640 1 -- 172.21.15.103:0/4211888670 shutdown_connections 2024-09-16T02:05:10.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:10.636+0000 7f49a8c00640 1 -- 172.21.15.103:0/4211888670 wait complete. 2024-09-16T02:05:10.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:10 smithi103 ceph-mon[30398]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:11.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:10 smithi136 ceph-mon[35323]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:11.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:10 smithi172 ceph-mon[35340]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:11.270 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T02:05:11.279 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-16T02:05:11.280 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T02:05:11.649 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:11.932 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:11 smithi103 ceph-mon[30398]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:05:11.932 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:11 smithi103 ceph-mon[30398]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:12.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:11 smithi136 ceph-mon[35323]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:05:12.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:11 smithi136 ceph-mon[35323]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:12.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:11 smithi172 ceph-mon[35340]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:05:12.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:11 smithi172 ceph-mon[35340]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:12.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.195+0000 7f90d7200640 1 -- 172.21.15.103:0/149073877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90d0105920 msgr2=0x7f90d010c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:12.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.195+0000 7f90d7200640 1 --2- 172.21.15.103:0/149073877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90d0105920 0x7f90d010c240 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009ca0 tx=0x7f90c802a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.195+0000 7f90d7200640 1 -- 172.21.15.103:0/149073877 shutdown_connections 2024-09-16T02:05:12.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.195+0000 7f90d7200640 1 --2- 172.21.15.103:0/149073877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90d0105920 0x7f90d010c240 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.195+0000 7f90d7200640 1 --2- 172.21.15.103:0/149073877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90d0104f60 0x7f90d01053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.195+0000 7f90d7200640 1 --2- 172.21.15.103:0/149073877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d0103e00 0x7f90d0104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.195+0000 7f90d7200640 1 -- 172.21.15.103:0/149073877 >> 172.21.15.103:0/149073877 conn(0x7f90d00ff5b0 msgr2=0x7f90d01019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:12.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.195+0000 7f90d7200640 1 -- 172.21.15.103:0/149073877 shutdown_connections 2024-09-16T02:05:12.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.195+0000 7f90d7200640 1 -- 172.21.15.103:0/149073877 wait complete. 2024-09-16T02:05:12.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.196+0000 7f90d7200640 1 Processor -- start 2024-09-16T02:05:12.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.197+0000 7f90d7200640 1 -- start start 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.197+0000 7f90d7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90d0103e00 0x7f90d019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.197+0000 7f90d7200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90d0104f60 0x7f90d019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.197+0000 7f90d7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d0105920 0x7f90d01a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.197+0000 7f90d7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90d010fae0 con 0x7f90d0103e00 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.197+0000 7f90d7200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f90d010f960 con 0x7f90d0104f60 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.197+0000 7f90d7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90d010fc60 con 0x7f90d0105920 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.197+0000 7f90cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90d0103e00 0x7f90d019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.197+0000 7f90cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90d0103e00 0x7f90d019a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49276/0 (socket says 172.21.15.103:49276) 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.197+0000 7f90cfe00640 1 -- 172.21.15.103:0/3894632825 learned_addr learned my addr 172.21.15.103:0/3894632825 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.198+0000 7f90cfe00640 1 -- 172.21.15.103:0/3894632825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d0105920 msgr2=0x7f90d01a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.198+0000 7f90cfe00640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d0105920 0x7f90d01a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.198+0000 7f90cfe00640 1 -- 172.21.15.103:0/3894632825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90d0104f60 msgr2=0x7f90d019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:12.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.198+0000 7f90cfe00640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90d0104f60 0x7f90d019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.198+0000 7f90cfe00640 1 -- 172.21.15.103:0/3894632825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d01a1c80 con 0x7f90d0103e00 2024-09-16T02:05:12.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.198+0000 7f90cfe00640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90d0103e00 0x7f90d019a050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e9f0 tx=0x7f90c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:12.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.198+0000 7f90ccc00640 1 -- 172.21.15.103:0/3894632825 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c000cd80 con 0x7f90d0103e00 2024-09-16T02:05:12.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.198+0000 7f90ccc00640 1 -- 172.21.15.103:0/3894632825 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c0010910 con 0x7f90d0103e00 2024-09-16T02:05:12.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.198+0000 7f90ccc00640 1 -- 172.21.15.103:0/3894632825 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0018990 con 0x7f90d0103e00 2024-09-16T02:05:12.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.198+0000 7f90d7200640 1 -- 172.21.15.103:0/3894632825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d01a1f70 con 0x7f90d0103e00 2024-09-16T02:05:12.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.199+0000 7f90d7200640 1 -- 172.21.15.103:0/3894632825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d01a23d0 con 0x7f90d0103e00 2024-09-16T02:05:12.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.200+0000 7f90d7200640 1 -- 172.21.15.103:0/3894632825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d010b120 con 0x7f90d0103e00 2024-09-16T02:05:12.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.201+0000 7f90ccc00640 1 -- 172.21.15.103:0/3894632825 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f90c0010ac0 con 0x7f90d0103e00 2024-09-16T02:05:12.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.203+0000 7f90ccc00640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90b40a3380 0x7f90b40a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:12.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.203+0000 7f90ccc00640 1 -- 172.21.15.103:0/3894632825 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f90c00cfe80 con 0x7f90d0103e00 2024-09-16T02:05:12.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.203+0000 7f90cf400640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90b40a3380 0x7f90b40a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:12.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.204+0000 7f90cf400640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90b40a3380 0x7f90b40a5840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f90bc005d00 tx=0x7f90bc005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.206+0000 7f90ccc00640 1 -- 172.21.15.103:0/3894632825 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90c009c9d0 con 0x7f90d0103e00 2024-09-16T02:05:12.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.418+0000 7f90d7200640 1 -- 172.21.15.103:0/3894632825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f90d002da70 con 0x7f90d0103e00 2024-09-16T02:05:12.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.418+0000 7f90ccc00640 1 -- 172.21.15.103:0/3894632825 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3103 (secure 0 0 0) 0x7f90c009c2d0 con 0x7f90d0103e00 2024-09-16T02:05:12.420 INFO:teuthology.orchestra.run.smithi103.stdout:e23 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout:btime 2024-09-16T02:03:50:098205+0000 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout:legacy client fscid: 1 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout:Filesystem 'cephfs' (1) 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout:fs_name cephfs 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 23 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-16T02:03:37.830568+0000 2024-09-16T02:05:12.421 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-16T02:03:49.812580+0000 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:tableserver 0 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:root 0 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:session_timeout 300 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:session_autoclose 300 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:max_file_size 1099511627776 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:max_xattr_size 65536 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:required_client_features {} 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure 0 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure_osd_epoch 0 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:max_mds 3 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:in 0,1,2 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:up {0=24545,1=24506,2=24505} 2024-09-16T02:05:12.422 INFO:teuthology.orchestra.run.smithi103.stdout:failed 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:damaged 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:stopped 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:data_pools [3] 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:metadata_pool 2 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:inline_data disabled 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:balancer 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:bal_rank_mask -1 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:standby_count_wanted 0 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:qdb_cluster leader: 24545 members: 24505,24506,24545 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.b{0:24545} state up:active seq 5 addr [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.c{0:24490} state up:standby-replay seq 1 addr [v2:172.21.15.172:6836/1584538248,v1:172.21.15.172:6837/1584538248] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.423 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.h{1:24506} state up:active seq 10 addr [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.f{1:24484} state up:standby-replay seq 1 addr [v2:172.21.15.172:6834/2720370264,v1:172.21.15.172:6835/2720370264] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.i{2:24505} state up:active seq 9 addr [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.e{2:24482} state up:standby-replay seq 1 addr [v2:172.21.15.136:6834/2566717089,v1:172.21.15.136:6835/2566717089] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout:Standby daemons: 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.j{-1:14457} state up:standby seq 1 addr [v2:172.21.15.103:6834/2357019382,v1:172.21.15.103:6835/2357019382] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.g{-1:14514} state up:standby seq 1 addr [v2:172.21.15.103:6838/3043303715,v1:172.21.15.103:6839/3043303715] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.a{-1:14526} state up:standby seq 1 addr [v2:172.21.15.103:6840/2552214171,v1:172.21.15.103:6841/2552214171] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.l{-1:24439} state up:standby seq 1 addr [v2:172.21.15.172:6832/1362058923,v1:172.21.15.172:6833/1362058923] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.k{-1:24452} state up:standby seq 1 addr [v2:172.21.15.136:6832/3604147096,v1:172.21.15.136:6833/3604147096] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.424 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.d{-1:24463} state up:standby seq 1 addr [v2:172.21.15.103:6836/2740838933,v1:172.21.15.103:6837/2740838933] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T02:05:12.425 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 23 2024-09-16T02:05:12.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 -- 172.21.15.103:0/3894632825 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90b40a3380 msgr2=0x7f90b40a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:12.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90b40a3380 0x7f90b40a5840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f90bc005d00 tx=0x7f90bc005c50 comp rx=0 tx=0).stop 2024-09-16T02:05:12.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 -- 172.21.15.103:0/3894632825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90d0103e00 msgr2=0x7f90d019a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:12.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90d0103e00 0x7f90d019a050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e9f0 tx=0x7f90c000eeb0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 -- 172.21.15.103:0/3894632825 shutdown_connections 2024-09-16T02:05:12.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90b40a3380 0x7f90b40a5840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d0105920 0x7f90d01a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90d0104f60 0x7f90d019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 --2- 172.21.15.103:0/3894632825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90d0103e00 0x7f90d019a050 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:12.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 -- 172.21.15.103:0/3894632825 >> 172.21.15.103:0/3894632825 conn(0x7f90d00ff5b0 msgr2=0x7f90d01082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:12.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 -- 172.21.15.103:0/3894632825 shutdown_connections 2024-09-16T02:05:12.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:12.422+0000 7f90d7200640 1 -- 172.21.15.103:0/3894632825 wait complete. 2024-09-16T02:05:12.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:12 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3894632825' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T02:05:13.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:12 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3894632825' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T02:05:13.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:12 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3894632825' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T02:05:13.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-16T02:05:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:13.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:13 smithi103 ceph-mon[30398]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:14.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:13 smithi136 ceph-mon[35323]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:14.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:13 smithi172 ceph-mon[35340]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:14.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.132+0000 7f9ec3e00640 1 -- 172.21.15.103:0/4063402397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4101a40 msgr2=0x7f9ec4101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.132+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/4063402397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4101a40 0x7f9ec4101e40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009900 tx=0x7f9eb402a410 comp rx=0 tx=0).stop 2024-09-16T02:05:14.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.132+0000 7f9ec3e00640 1 -- 172.21.15.103:0/4063402397 shutdown_connections 2024-09-16T02:05:14.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.132+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/4063402397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec4103600 0x7f9ec4109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.132+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/4063402397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec4102c40 0x7f9ec41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.132+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/4063402397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4101a40 0x7f9ec4101e40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.132+0000 7f9ec3e00640 1 -- 172.21.15.103:0/4063402397 >> 172.21.15.103:0/4063402397 conn(0x7f9ec40fd1f0 msgr2=0x7f9ec40ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:14.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.132+0000 7f9ec3e00640 1 -- 172.21.15.103:0/4063402397 shutdown_connections 2024-09-16T02:05:14.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.132+0000 7f9ec3e00640 1 -- 172.21.15.103:0/4063402397 wait complete. 2024-09-16T02:05:14.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.133+0000 7f9ec3e00640 1 Processor -- start 2024-09-16T02:05:14.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.133+0000 7f9ec3e00640 1 -- start start 2024-09-16T02:05:14.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4101a40 0x7f9ec4199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec4102c40 0x7f9ec419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec4103600 0x7f9ec41a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec3e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec410d720 con 0x7f9ec4101a40 2024-09-16T02:05:14.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec3e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec410d5a0 con 0x7f9ec4103600 2024-09-16T02:05:14.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec410d8a0 con 0x7f9ec4102c40 2024-09-16T02:05:14.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4101a40 0x7f9ec4199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4101a40 0x7f9ec4199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49308/0 (socket says 172.21.15.103:49308) 2024-09-16T02:05:14.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec2a00640 1 -- 172.21.15.103:0/175227741 learned_addr learned my addr 172.21.15.103:0/175227741 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:14.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec3400640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec4103600 0x7f9ec41a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec2a00640 1 -- 172.21.15.103:0/175227741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec4102c40 msgr2=0x7f9ec419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec2a00640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec4102c40 0x7f9ec419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec2a00640 1 -- 172.21.15.103:0/175227741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec4103600 msgr2=0x7f9ec41a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec2a00640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec4103600 0x7f9ec41a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.134+0000 7f9ec2a00640 1 -- 172.21.15.103:0/175227741 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec41a1b90 con 0x7f9ec4101a40 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.135+0000 7f9ec2a00640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4101a40 0x7f9ec4199f60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4031610 tx=0x7f9eb40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.135+0000 7f9ebb400640 1 -- 172.21.15.103:0/175227741 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb403f030 con 0x7f9ec4101a40 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.135+0000 7f9ebb400640 1 -- 172.21.15.103:0/175227741 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4003ce0 con 0x7f9ec4101a40 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.135+0000 7f9ebb400640 1 -- 172.21.15.103:0/175227741 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4035070 con 0x7f9ec4101a40 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.135+0000 7f9ec3e00640 1 -- 172.21.15.103:0/175227741 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec41a1e20 con 0x7f9ec4101a40 2024-09-16T02:05:14.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.135+0000 7f9ec3e00640 1 -- 172.21.15.103:0/175227741 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec41a22a0 con 0x7f9ec4101a40 2024-09-16T02:05:14.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.136+0000 7f9ec3e00640 1 -- 172.21.15.103:0/175227741 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec41a24d0 con 0x7f9ec4101a40 2024-09-16T02:05:14.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.138+0000 7f9ebb400640 1 -- 172.21.15.103:0/175227741 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9eb4042030 con 0x7f9ec4101a40 2024-09-16T02:05:14.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.139+0000 7f9ebb400640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3300 0x7f9ea80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:14.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.139+0000 7f9ebb400640 1 -- 172.21.15.103:0/175227741 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f9eb40f0080 con 0x7f9ec4101a40 2024-09-16T02:05:14.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.140+0000 7f9ec2000640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3300 0x7f9ea80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:14.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.140+0000 7f9ec2000640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3300 0x7f9ea80a57c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ec419b480 tx=0x7f9eb0006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:14.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.143+0000 7f9ebb400640 1 -- 172.21.15.103:0/175227741 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9eb40bcbd0 con 0x7f9ec4101a40 2024-09-16T02:05:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.319+0000 7f9ec3e00640 1 -- 172.21.15.103:0/175227741 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f9ec41a28c0 con 0x7f9ec4101a40 2024-09-16T02:05:14.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.320+0000 7f9ebb400640 1 -- 172.21.15.103:0/175227741 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v79) ==== 56+0+5345 (secure 0 0 0) 0x7f9eb40bc4d0 con 0x7f9ec4101a40 2024-09-16T02:05:14.323 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 79 2024-09-16T02:05:14.323 INFO:teuthology.orchestra.run.smithi103.stdout:fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T02:05:14.323 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-16T01:55:37.667040+0000 2024-09-16T02:05:14.323 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-16T02:04:29.624699+0000 2024-09-16T02:05:14.323 INFO:teuthology.orchestra.run.smithi103.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T02:05:14.323 INFO:teuthology.orchestra.run.smithi103.stdout:crush_version 26 2024-09-16T02:05:14.323 INFO:teuthology.orchestra.run.smithi103.stdout:full_ratio 0.95 2024-09-16T02:05:14.323 INFO:teuthology.orchestra.run.smithi103.stdout:backfillfull_ratio 0.9 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:nearfull_ratio 0.85 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:require_min_compat_client luminous 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:min_compat_client luminous 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:require_osd_release squid 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:stretch_mode_enabled false 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:max_osd 12 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6802/484703095,v1:172.21.15.103:6803/484703095] [v2:172.21.15.103:6804/484703095,v1:172.21.15.103:6805/484703095] exists,up 839e2a76-348d-4d7f-b43b-f446e829aade 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 up in weight 1 up_from 12 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/1430190507,v1:172.21.15.136:6801/1430190507] [v2:172.21.15.136:6802/1430190507,v1:172.21.15.136:6803/1430190507] exists,up 393999bb-cc86-41ed-b0fa-7ae957c11374 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 up in weight 1 up_from 16 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6800/544248973,v1:172.21.15.172:6801/544248973] [v2:172.21.15.172:6802/544248973,v1:172.21.15.172:6803/544248973] exists,up a337ce9d-76c5-4410-b843-fce557c1401e 2024-09-16T02:05:14.324 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 up in weight 1 up_from 22 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6810/2129636883,v1:172.21.15.103:6811/2129636883] [v2:172.21.15.103:6812/2129636883,v1:172.21.15.103:6813/2129636883] exists,up c2735e5b-dd14-483d-84de-a0172d862b73 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 up in weight 1 up_from 29 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/2555490296,v1:172.21.15.136:6809/2555490296] [v2:172.21.15.136:6810/2555490296,v1:172.21.15.136:6811/2555490296] exists,up 35714991-3234-4719-b05f-604de810aac7 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 up in weight 1 up_from 34 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6808/1913335537,v1:172.21.15.172:6809/1913335537] [v2:172.21.15.172:6810/1913335537,v1:172.21.15.172:6811/1913335537] exists,up f17f3197-8dc3-4e97-b7ee-a36975327aa7 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 up in weight 1 up_from 41 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6818/1754743602,v1:172.21.15.103:6819/1754743602] [v2:172.21.15.103:6820/1754743602,v1:172.21.15.103:6821/1754743602] exists,up 8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 up in weight 1 up_from 46 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/2248125128,v1:172.21.15.136:6817/2248125128] [v2:172.21.15.136:6818/2248125128,v1:172.21.15.136:6819/2248125128] exists,up a8c4b067-a2b3-404e-867e-8c8e51ecd4b2 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 up in weight 1 up_from 53 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6816/1611593335,v1:172.21.15.172:6817/1611593335] [v2:172.21.15.172:6818/1611593335,v1:172.21.15.172:6819/1611593335] exists,up b835bf50-420f-42f9-8ca5-c3f1527be672 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 up in weight 1 up_from 60 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6826/1291384117,v1:172.21.15.103:6827/1291384117] [v2:172.21.15.103:6828/1291384117,v1:172.21.15.103:6829/1291384117] exists,up 46691205-fd3d-4f55-92bf-eae5bc09f6d7 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 up in weight 1 up_from 64 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/3121812139,v1:172.21.15.136:6825/3121812139] [v2:172.21.15.136:6826/3121812139,v1:172.21.15.136:6827/3121812139] exists,up f3f31571-16de-4ef9-a951-f607776400a2 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 up in weight 1 up_from 71 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6824/2746302927,v1:172.21.15.172:6825/2746302927] [v2:172.21.15.172:6826/2746302927,v1:172.21.15.172:6827/2746302927] exists,up 84a838c3-7bec-402f-a65a-78030b0380d7 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.0 [8,4] 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.f [5,1] 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.10 [8,11] 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.16 [2,4] 2024-09-16T02:05:14.325 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.1a [10,11] 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.1b [2,4] 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.1d [2,1,7,6] 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.21 [4,6] 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.23 [4,8] 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1646778488 expires 2024-09-17T01:56:28.359910+0000 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1801195624 expires 2024-09-17T01:56:28.359910+0000 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1537689916 expires 2024-09-17T01:56:28.359910+0000 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/623307529 expires 2024-09-17T01:56:28.359910+0000 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/1247798496 expires 2024-09-17T01:56:01.936729+0000 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/623307529 expires 2024-09-17T01:56:28.359910+0000 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/1247798496 expires 2024-09-17T01:56:01.936729+0000 2024-09-16T02:05:14.326 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/2568354020 expires 2024-09-17T01:56:01.936729+0000 2024-09-16T02:05:14.327 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1667549073 expires 2024-09-17T01:56:01.936729+0000 2024-09-16T02:05:14.327 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/3586311435 expires 2024-09-17T01:56:01.936729+0000 2024-09-16T02:05:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.322+0000 7f9ec3e00640 1 -- 172.21.15.103:0/175227741 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3300 msgr2=0x7f9ea80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.322+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3300 0x7f9ea80a57c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ec419b480 tx=0x7f9eb0006c90 comp rx=0 tx=0).stop 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.322+0000 7f9ec3e00640 1 -- 172.21.15.103:0/175227741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4101a40 msgr2=0x7f9ec4199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.322+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4101a40 0x7f9ec4199f60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4031610 tx=0x7f9eb40043b0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.322+0000 7f9ec3e00640 1 -- 172.21.15.103:0/175227741 shutdown_connections 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.322+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3300 0x7f9ea80a57c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.322+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec4103600 0x7f9ec41a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.322+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec4102c40 0x7f9ec419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.322+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/175227741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4101a40 0x7f9ec4199f60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.322+0000 7f9ec3e00640 1 -- 172.21.15.103:0/175227741 >> 172.21.15.103:0/175227741 conn(0x7f9ec40fd1f0 msgr2=0x7f9ec4108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.323+0000 7f9ec3e00640 1 -- 172.21.15.103:0/175227741 shutdown_connections 2024-09-16T02:05:14.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:14.323+0000 7f9ec3e00640 1 -- 172.21.15.103:0/175227741 wait complete. 2024-09-16T02:05:14.850 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:14 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/175227741' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T02:05:15.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:14 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/175227741' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T02:05:15.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:14 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/175227741' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T02:05:15.086 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-16T02:05:15.099 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:05:15.467 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:15.943 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:15 smithi103 ceph-mon[30398]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:16.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:15 smithi136 ceph-mon[35323]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:15 smithi172 ceph-mon[35340]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:16.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.050+0000 7fac7aa00640 1 -- 172.21.15.103:0/2293818881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac74103e30 msgr2=0x7fac74104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:16.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.050+0000 7fac7aa00640 1 --2- 172.21.15.103:0/2293818881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac74103e30 0x7fac74104210 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fac68009900 tx=0x7fac6802a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.050+0000 7fac7aa00640 1 -- 172.21.15.103:0/2293818881 shutdown_connections 2024-09-16T02:05:16.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.050+0000 7fac7aa00640 1 --2- 172.21.15.103:0/2293818881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac74108f60 0x7fac74110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.050+0000 7fac7aa00640 1 --2- 172.21.15.103:0/2293818881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac741047e0 0x7fac74108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.050+0000 7fac7aa00640 1 --2- 172.21.15.103:0/2293818881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac74103e30 0x7fac74104210 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.050+0000 7fac7aa00640 1 -- 172.21.15.103:0/2293818881 >> 172.21.15.103:0/2293818881 conn(0x7fac740ff8d0 msgr2=0x7fac74101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:16.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.050+0000 7fac7aa00640 1 -- 172.21.15.103:0/2293818881 shutdown_connections 2024-09-16T02:05:16.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.050+0000 7fac7aa00640 1 -- 172.21.15.103:0/2293818881 wait complete. 2024-09-16T02:05:16.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.051+0000 7fac7aa00640 1 Processor -- start 2024-09-16T02:05:16.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.051+0000 7fac7aa00640 1 -- start start 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.051+0000 7fac7aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac74103e30 0x7fac7419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.051+0000 7fac7aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac741047e0 0x7fac7419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.051+0000 7fac7aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac74108f60 0x7fac7419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.051+0000 7fac7aa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac74113fc0 con 0x7fac741047e0 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.051+0000 7fac7aa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac74113e40 con 0x7fac74103e30 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.051+0000 7fac7aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fac74114140 con 0x7fac74108f60 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.051+0000 7fac78c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac741047e0 0x7fac7419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.052+0000 7fac78c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac741047e0 0x7fac7419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49322/0 (socket says 172.21.15.103:49322) 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.052+0000 7fac78c00640 1 -- 172.21.15.103:0/869078574 learned_addr learned my addr 172.21.15.103:0/869078574 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.052+0000 7fac79600640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac74103e30 0x7fac7419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.052+0000 7fac7a000640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac74108f60 0x7fac7419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.052+0000 7fac78c00640 1 -- 172.21.15.103:0/869078574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac74108f60 msgr2=0x7fac7419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.052+0000 7fac78c00640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac74108f60 0x7fac7419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.052+0000 7fac78c00640 1 -- 172.21.15.103:0/869078574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac74103e30 msgr2=0x7fac7419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:16.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.052+0000 7fac78c00640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac74103e30 0x7fac7419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.052+0000 7fac78c00640 1 -- 172.21.15.103:0/869078574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7419f430 con 0x7fac741047e0 2024-09-16T02:05:16.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.052+0000 7fac78c00640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac741047e0 0x7fac7419a9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fac6000d770 tx=0x7fac6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:16.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.053+0000 7fac72000640 1 -- 172.21.15.103:0/869078574 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60017070 con 0x7fac741047e0 2024-09-16T02:05:16.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.053+0000 7fac72000640 1 -- 172.21.15.103:0/869078574 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac60005500 con 0x7fac741047e0 2024-09-16T02:05:16.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.053+0000 7fac7aa00640 1 -- 172.21.15.103:0/869078574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7419f720 con 0x7fac741047e0 2024-09-16T02:05:16.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.053+0000 7fac72000640 1 -- 172.21.15.103:0/869078574 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60004e00 con 0x7fac741047e0 2024-09-16T02:05:16.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.053+0000 7fac7aa00640 1 -- 172.21.15.103:0/869078574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac741a7000 con 0x7fac741047e0 2024-09-16T02:05:16.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.054+0000 7fac7aa00640 1 -- 172.21.15.103:0/869078574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac74104210 con 0x7fac741047e0 2024-09-16T02:05:16.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.056+0000 7fac72000640 1 -- 172.21.15.103:0/869078574 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fac60004fa0 con 0x7fac741047e0 2024-09-16T02:05:16.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.058+0000 7fac72000640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fac5c0a32d0 0x7fac5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:16.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.059+0000 7fac72000640 1 -- 172.21.15.103:0/869078574 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fac600cf490 con 0x7fac741047e0 2024-09-16T02:05:16.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.059+0000 7fac79600640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fac5c0a32d0 0x7fac5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:16.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.059+0000 7fac79600640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fac5c0a32d0 0x7fac5c0a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fac68009230 tx=0x7fac680058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:16.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.062+0000 7fac72000640 1 -- 172.21.15.103:0/869078574 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac6009b990 con 0x7fac741047e0 2024-09-16T02:05:16.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.271+0000 7fac7aa00640 1 -- 172.21.15.103:0/869078574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac7410bbb0 con 0x7fac741047e0 2024-09-16T02:05:16.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.272+0000 7fac72000640 1 -- 172.21.15.103:0/869078574 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7fac6009fc90 con 0x7fac741047e0 2024-09-16T02:05:16.274 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:16.274 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":23,"btime":"2024-09-16T02:03:50:098205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:03:49.812580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24505,24506,24545]},"id":1}]} 2024-09-16T02:05:16.275 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 23 2024-09-16T02:05:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.274+0000 7fac7aa00640 1 -- 172.21.15.103:0/869078574 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fac5c0a32d0 msgr2=0x7fac5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.274+0000 7fac7aa00640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fac5c0a32d0 0x7fac5c0a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fac68009230 tx=0x7fac680058d0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.274+0000 7fac7aa00640 1 -- 172.21.15.103:0/869078574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac741047e0 msgr2=0x7fac7419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.274+0000 7fac7aa00640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac741047e0 0x7fac7419a9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fac6000d770 tx=0x7fac6000dc30 comp rx=0 tx=0).stop 2024-09-16T02:05:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.274+0000 7fac7aa00640 1 -- 172.21.15.103:0/869078574 shutdown_connections 2024-09-16T02:05:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.274+0000 7fac7aa00640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fac5c0a32d0 0x7fac5c0a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.274+0000 7fac7aa00640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac74108f60 0x7fac7419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.274+0000 7fac7aa00640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac741047e0 0x7fac7419a9c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.274+0000 7fac7aa00640 1 --2- 172.21.15.103:0/869078574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac74103e30 0x7fac7419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.274+0000 7fac7aa00640 1 -- 172.21.15.103:0/869078574 >> 172.21.15.103:0/869078574 conn(0x7fac740ff8d0 msgr2=0x7fac74100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:16.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.275+0000 7fac7aa00640 1 -- 172.21.15.103:0/869078574 shutdown_connections 2024-09-16T02:05:16.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:16.275+0000 7fac7aa00640 1 -- 172.21.15.103:0/869078574 wait complete. 2024-09-16T02:05:17.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:16 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/869078574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:17.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:16 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/869078574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:17.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:16 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/869078574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:17.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:05:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:17.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:17 smithi103 ceph-mon[30398]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.954+0000 7fb607a00640 1 -- 172.21.15.103:0/893801234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6001027a0 msgr2=0x7fb6001067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.954+0000 7fb607a00640 1 --2- 172.21.15.103:0/893801234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6001027a0 0x7fb6001067f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009900 tx=0x7fb5ec02a410 comp rx=0 tx=0).stop 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.954+0000 7fb607a00640 1 -- 172.21.15.103:0/893801234 shutdown_connections 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.954+0000 7fb607a00640 1 --2- 172.21.15.103:0/893801234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb600106f20 0x7fb60010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.954+0000 7fb607a00640 1 --2- 172.21.15.103:0/893801234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6001027a0 0x7fb6001067f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.954+0000 7fb607a00640 1 --2- 172.21.15.103:0/893801234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb600101df0 0x7fb6001021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.954+0000 7fb607a00640 1 -- 172.21.15.103:0/893801234 >> 172.21.15.103:0/893801234 conn(0x7fb6000fd8b0 msgr2=0x7fb6000ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.954+0000 7fb607a00640 1 -- 172.21.15.103:0/893801234 shutdown_connections 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.954+0000 7fb607a00640 1 -- 172.21.15.103:0/893801234 wait complete. 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.955+0000 7fb607a00640 1 Processor -- start 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.955+0000 7fb607a00640 1 -- start start 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.955+0000 7fb607a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb600101df0 0x7fb60019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.955+0000 7fb607a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6001027a0 0x7fb60019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.955+0000 7fb607a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb600106f20 0x7fb60019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.955+0000 7fb607a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb600112010 con 0x7fb600101df0 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.955+0000 7fb607a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb600111e90 con 0x7fb6001027a0 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.955+0000 7fb607a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb600112190 con 0x7fb600106f20 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb605000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb600101df0 0x7fb60019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb605000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb600101df0 0x7fb60019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45006/0 (socket says 172.21.15.103:45006) 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb605000640 1 -- 172.21.15.103:0/2667731442 learned_addr learned my addr 172.21.15.103:0/2667731442 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb605000640 1 -- 172.21.15.103:0/2667731442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb600106f20 msgr2=0x7fb60019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb605000640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb600106f20 0x7fb60019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb605000640 1 -- 172.21.15.103:0/2667731442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6001027a0 msgr2=0x7fb60019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb605000640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6001027a0 0x7fb60019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb605000640 1 -- 172.21.15.103:0/2667731442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60019f640 con 0x7fb600101df0 2024-09-16T02:05:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb605000640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb600101df0 0x7fb60019a690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b4a0 tx=0x7fb5f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb5fd600640 1 -- 172.21.15.103:0/2667731442 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4014070 con 0x7fb600101df0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb5fd600640 1 -- 172.21.15.103:0/2667731442 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f4010af0 con 0x7fb600101df0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.956+0000 7fb607a00640 1 -- 172.21.15.103:0/2667731442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60019f930 con 0x7fb600101df0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.957+0000 7fb5fd600640 1 -- 172.21.15.103:0/2667731442 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4017d10 con 0x7fb600101df0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.957+0000 7fb607a00640 1 -- 172.21.15.103:0/2667731442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6001a7210 con 0x7fb600101df0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.958+0000 7fb607a00640 1 -- 172.21.15.103:0/2667731442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6001021d0 con 0x7fb600101df0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.959+0000 7fb5fd600640 1 -- 172.21.15.103:0/2667731442 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb5f4017070 con 0x7fb600101df0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.961+0000 7fb5fd600640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5e80a32d0 0x7fb5e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.961+0000 7fb5fd600640 1 -- 172.21.15.103:0/2667731442 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fb5f40cee80 con 0x7fb600101df0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.961+0000 7fb5ffe00640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5e80a32d0 0x7fb5e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.962+0000 7fb5ffe00640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5e80a32d0 0x7fb5e80a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0096c0 tx=0x7fb5ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:17.965+0000 7fb5fd600640 1 -- 172.21.15.103:0/2667731442 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5f409bae0 con 0x7fb600101df0 2024-09-16T02:05:18.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:17 smithi136 ceph-mon[35323]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:18.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:17 smithi172 ceph-mon[35340]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:18.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.175+0000 7fb607a00640 1 -- 172.21.15.103:0/2667731442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb60019b9f0 con 0x7fb600101df0 2024-09-16T02:05:18.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.176+0000 7fb5fd600640 1 -- 172.21.15.103:0/2667731442 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7fb5f409b3e0 con 0x7fb600101df0 2024-09-16T02:05:18.179 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:18.179 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":23,"btime":"2024-09-16T02:03:50:098205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:03:49.812580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24505,24506,24545]},"id":1}]} 2024-09-16T02:05:18.180 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 23 2024-09-16T02:05:18.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 -- 172.21.15.103:0/2667731442 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5e80a32d0 msgr2=0x7fb5e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:18.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5e80a32d0 0x7fb5e80a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0096c0 tx=0x7fb5ec000bc0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 -- 172.21.15.103:0/2667731442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb600101df0 msgr2=0x7fb60019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:18.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb600101df0 0x7fb60019a690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b4a0 tx=0x7fb5f400b960 comp rx=0 tx=0).stop 2024-09-16T02:05:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 -- 172.21.15.103:0/2667731442 shutdown_connections 2024-09-16T02:05:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb5e80a32d0 0x7fb5e80a5790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb600106f20 0x7fb60019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6001027a0 0x7fb60019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 --2- 172.21.15.103:0/2667731442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb600101df0 0x7fb60019a690 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:18.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 -- 172.21.15.103:0/2667731442 >> 172.21.15.103:0/2667731442 conn(0x7fb6000fd8b0 msgr2=0x7fb6000ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:18.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 -- 172.21.15.103:0/2667731442 shutdown_connections 2024-09-16T02:05:18.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:18.179+0000 7fb607a00640 1 -- 172.21.15.103:0/2667731442 wait complete. 2024-09-16T02:05:18.948 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:18 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2667731442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:18.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:05:19.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:18 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2667731442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:18 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2667731442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:19.359 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:19.750 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:19 smithi103 ceph-mon[30398]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:19.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.852+0000 7ffa13200640 1 -- 172.21.15.103:0/1719397134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa14108f60 msgr2=0x7ffa14110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:19.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.852+0000 7ffa13200640 1 --2- 172.21.15.103:0/1719397134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa14108f60 0x7ffa14110ae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009be0 tx=0x7ffa0c02a6c0 comp rx=0 tx=0).stop 2024-09-16T02:05:19.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.852+0000 7ffa13200640 1 -- 172.21.15.103:0/1719397134 shutdown_connections 2024-09-16T02:05:19.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.852+0000 7ffa13200640 1 --2- 172.21.15.103:0/1719397134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa14108f60 0x7ffa14110ae0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:19.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.852+0000 7ffa13200640 1 --2- 172.21.15.103:0/1719397134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa141047e0 0x7ffa14108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:19.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.852+0000 7ffa13200640 1 --2- 172.21.15.103:0/1719397134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa14103e30 0x7ffa14104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:19.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.852+0000 7ffa13200640 1 -- 172.21.15.103:0/1719397134 >> 172.21.15.103:0/1719397134 conn(0x7ffa140ff8d0 msgr2=0x7ffa14101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:19.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.852+0000 7ffa13200640 1 -- 172.21.15.103:0/1719397134 shutdown_connections 2024-09-16T02:05:19.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.852+0000 7ffa13200640 1 -- 172.21.15.103:0/1719397134 wait complete. 2024-09-16T02:05:19.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.853+0000 7ffa13200640 1 Processor -- start 2024-09-16T02:05:19.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.853+0000 7ffa13200640 1 -- start start 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.853+0000 7ffa13200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa14103e30 0x7ffa1419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.853+0000 7ffa13200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa141047e0 0x7ffa1419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.853+0000 7ffa13200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa14108f60 0x7ffa1419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.853+0000 7ffa13200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14113f70 con 0x7ffa14103e30 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.853+0000 7ffa13200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14113df0 con 0x7ffa14108f60 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.853+0000 7ffa13200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141140f0 con 0x7ffa141047e0 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa11e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa14103e30 0x7ffa1419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa11e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa14103e30 0x7ffa1419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45024/0 (socket says 172.21.15.103:45024) 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa11e00640 1 -- 172.21.15.103:0/3520284285 learned_addr learned my addr 172.21.15.103:0/3520284285 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa09400640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa141047e0 0x7ffa1419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa12800640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa14108f60 0x7ffa1419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:19.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa11e00640 1 -- 172.21.15.103:0/3520284285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa141047e0 msgr2=0x7ffa1419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa11e00640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa141047e0 0x7ffa1419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa11e00640 1 -- 172.21.15.103:0/3520284285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa14108f60 msgr2=0x7ffa1419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa11e00640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa14108f60 0x7ffa1419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa11e00640 1 -- 172.21.15.103:0/3520284285 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1419f3e0 con 0x7ffa14103e30 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa11e00640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa14103e30 0x7ffa1419a430 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400e9f0 tx=0x7ffa0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.854+0000 7ffa0b400640 1 -- 172.21.15.103:0/3520284285 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400ccc0 con 0x7ffa14103e30 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.855+0000 7ffa0b400640 1 -- 172.21.15.103:0/3520284285 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa04010910 con 0x7ffa14103e30 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.855+0000 7ffa13200640 1 -- 172.21.15.103:0/3520284285 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1419f6d0 con 0x7ffa14103e30 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.855+0000 7ffa0b400640 1 -- 172.21.15.103:0/3520284285 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa040188a0 con 0x7ffa14103e30 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.855+0000 7ffa13200640 1 -- 172.21.15.103:0/3520284285 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa141a6fb0 con 0x7ffa14103e30 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.856+0000 7ffa13200640 1 -- 172.21.15.103:0/3520284285 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa14104210 con 0x7ffa14103e30 2024-09-16T02:05:19.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.858+0000 7ffa0b400640 1 -- 172.21.15.103:0/3520284285 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ffa04010ac0 con 0x7ffa14103e30 2024-09-16T02:05:19.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.859+0000 7ffa0b400640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f80a32d0 0x7ff9f80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:19.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.859+0000 7ffa0b400640 1 -- 172.21.15.103:0/3520284285 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7ffa040cff00 con 0x7ffa14103e30 2024-09-16T02:05:19.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.859+0000 7ffa09400640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f80a32d0 0x7ff9f80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:19.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.860+0000 7ffa09400640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f80a32d0 0x7ff9f80a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa140ffcf0 tx=0x7ff9fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:19.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:19.863+0000 7ffa0b400640 1 -- 172.21.15.103:0/3520284285 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa0409ca50 con 0x7ffa14103e30 2024-09-16T02:05:20.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:19 smithi136 ceph-mon[35323]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:20.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.041+0000 7ffa13200640 1 -- 172.21.15.103:0/3520284285 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa1410bc10 con 0x7ffa14103e30 2024-09-16T02:05:20.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.042+0000 7ffa0b400640 1 -- 172.21.15.103:0/3520284285 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22861 (secure 0 0 0) 0x7ffa0409c350 con 0x7ffa14103e30 2024-09-16T02:05:20.045 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":79,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:04:29.624699+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":10,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":7,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:05:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:19 smithi172 ceph-mon[35340]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:20.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 -- 172.21.15.103:0/3520284285 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f80a32d0 msgr2=0x7ff9f80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:20.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f80a32d0 0x7ff9f80a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa140ffcf0 tx=0x7ff9fc009260 comp rx=0 tx=0).stop 2024-09-16T02:05:20.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 -- 172.21.15.103:0/3520284285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa14103e30 msgr2=0x7ffa1419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:20.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa14103e30 0x7ffa1419a430 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400e9f0 tx=0x7ffa0400edf0 comp rx=0 tx=0).stop 2024-09-16T02:05:20.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 -- 172.21.15.103:0/3520284285 shutdown_connections 2024-09-16T02:05:20.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f80a32d0 0x7ff9f80a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:20.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa14108f60 0x7ffa1419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:20.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa141047e0 0x7ffa1419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:20.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 --2- 172.21.15.103:0/3520284285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa14103e30 0x7ffa1419a430 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:20.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 -- 172.21.15.103:0/3520284285 >> 172.21.15.103:0/3520284285 conn(0x7ffa140ff8d0 msgr2=0x7ffa14100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:20.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 -- 172.21.15.103:0/3520284285 shutdown_connections 2024-09-16T02:05:20.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:20.045+0000 7ffa13200640 1 -- 172.21.15.103:0/3520284285 wait complete. 2024-09-16T02:05:20.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:20 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3520284285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:20.815 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T02:03:37.830568+0000', 'modified': '2024-09-16T02:03:49.812580+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24545, 'mds_1': 24506, 'mds_2': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/2566717089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 2566717089}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 2566717089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/2720370264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 2720370264}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2720370264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6837/1584538248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 1584538248}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 1584538248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.172:6839/3474994563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 3474994563}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 3474994563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.136:6837/236571207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 236571207}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 236571207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.136:6839/3041647128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 3041647128}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 3041647128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24505, 24506, 24545]} 2024-09-16T02:05:20.815 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-16T02:05:20.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T02:05:21.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:20 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3520284285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:21.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:20 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3520284285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:21.187 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:21.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.750+0000 7fe391000640 1 -- 172.21.15.103:0/399795928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe38c106f10 msgr2=0x7fe38c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:21.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.750+0000 7fe391000640 1 --2- 172.21.15.103:0/399795928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe38c106f10 0x7fe38c10ea90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe384009ca0 tx=0x7fe38402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.750+0000 7fe391000640 1 -- 172.21.15.103:0/399795928 shutdown_connections 2024-09-16T02:05:21.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.750+0000 7fe391000640 1 --2- 172.21.15.103:0/399795928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe38c106f10 0x7fe38c10ea90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.750+0000 7fe391000640 1 --2- 172.21.15.103:0/399795928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe38c102790 0x7fe38c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.750+0000 7fe391000640 1 --2- 172.21.15.103:0/399795928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c101de0 0x7fe38c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.750+0000 7fe391000640 1 -- 172.21.15.103:0/399795928 >> 172.21.15.103:0/399795928 conn(0x7fe38c0fd8c0 msgr2=0x7fe38c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:21.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.750+0000 7fe391000640 1 -- 172.21.15.103:0/399795928 shutdown_connections 2024-09-16T02:05:21.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.750+0000 7fe391000640 1 -- 172.21.15.103:0/399795928 wait complete. 2024-09-16T02:05:21.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.751+0000 7fe391000640 1 Processor -- start 2024-09-16T02:05:21.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.751+0000 7fe391000640 1 -- start start 2024-09-16T02:05:21.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.751+0000 7fe391000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe38c101de0 0x7fe38c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.751+0000 7fe391000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c102790 0x7fe38c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe391000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe38c106f10 0x7fe38c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe391000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c111f70 con 0x7fe38c106f10 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe391000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c111df0 con 0x7fe38c101de0 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe391000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c1120f0 con 0x7fe38c102790 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe38a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe38c106f10 0x7fe38c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe38a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe38c106f10 0x7fe38c19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45046/0 (socket says 172.21.15.103:45046) 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe38a800640 1 -- 172.21.15.103:0/1353887913 learned_addr learned my addr 172.21.15.103:0/1353887913 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe389400640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c102790 0x7fe38c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe38a800640 1 -- 172.21.15.103:0/1353887913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c102790 msgr2=0x7fe38c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe38a800640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c102790 0x7fe38c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe38a800640 1 -- 172.21.15.103:0/1353887913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe38c101de0 msgr2=0x7fe38c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe38a800640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe38c101de0 0x7fe38c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.752+0000 7fe38a800640 1 -- 172.21.15.103:0/1353887913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c19f5a0 con 0x7fe38c106f10 2024-09-16T02:05:21.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.753+0000 7fe38a800640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe38c106f10 0x7fe38c19eec0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe384031a20 tx=0x7fe384031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:21.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.753+0000 7fe382a00640 1 -- 172.21.15.103:0/1353887913 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384003b70 con 0x7fe38c106f10 2024-09-16T02:05:21.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.753+0000 7fe382a00640 1 -- 172.21.15.103:0/1353887913 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe384003d10 con 0x7fe38c106f10 2024-09-16T02:05:21.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.753+0000 7fe391000640 1 -- 172.21.15.103:0/1353887913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38c19f830 con 0x7fe38c106f10 2024-09-16T02:05:21.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.753+0000 7fe382a00640 1 -- 172.21.15.103:0/1353887913 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38402fc40 con 0x7fe38c106f10 2024-09-16T02:05:21.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.753+0000 7fe391000640 1 -- 172.21.15.103:0/1353887913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38c1a7170 con 0x7fe38c106f10 2024-09-16T02:05:21.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.756+0000 7fe382a00640 1 -- 172.21.15.103:0/1353887913 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe384004050 con 0x7fe38c106f10 2024-09-16T02:05:21.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.756+0000 7fe393a01740 1 Processor -- start 2024-09-16T02:05:21.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.756+0000 7fe393a01740 1 -- start start 2024-09-16T02:05:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe393a01740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fc330 0x564ee15fc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe393a01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1606fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe393a01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1609900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe393a01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564ee15620b0 con 0x564ee1607510 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe393a01740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ee1509ab0 con 0x564ee15fc330 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe393a01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564ee155e070 con 0x564ee15fcc50 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe389400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fc330 0x564ee15fc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe389e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1606fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe389400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1609900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe389e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1606fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:35190/0 (socket says 172.21.15.103:35190) 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe389e00640 1 -- 172.21.15.103:0/3087083398 learned_addr learned my addr 172.21.15.103:0/3087083398 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe389400640 1 --2- 172.21.15.103:0/3087083398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1609900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45058/0 (socket says 172.21.15.103:45058) 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe389e00640 1 -- 172.21.15.103:0/3087083398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fc330 msgr2=0x564ee15fc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe389e00640 1 --2- 172.21.15.103:0/3087083398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fc330 0x564ee15fc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe389e00640 1 -- 172.21.15.103:0/3087083398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 msgr2=0x564ee1609900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:21.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.757+0000 7fe389e00640 1 --2- 172.21.15.103:0/3087083398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1609900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.758+0000 7fe389e00640 1 -- 172.21.15.103:0/3087083398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee15164a0 con 0x564ee15fcc50 2024-09-16T02:05:21.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.758+0000 7fe389400640 1 --2- 172.21.15.103:0/3087083398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1609900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:21.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.758+0000 7fe382a00640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3700a3350 0x7fe3700a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.758+0000 7fe389e00640 1 --2- 172.21.15.103:0/3087083398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1606fd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009c40 tx=0x7fe37c00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26a7be5b02e64241 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:21.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.758+0000 7fe382a00640 1 -- 172.21.15.103:0/1353887913 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fe3840f0da0 con 0x7fe38c106f10 2024-09-16T02:05:21.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.758+0000 7fe389400640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3700a3350 0x7fe3700a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.758+0000 7fe377400640 1 -- 172.21.15.103:0/3087083398 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c005020 con 0x564ee15fcc50 2024-09-16T02:05:21.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.758+0000 7fe377400640 1 -- 172.21.15.103:0/3087083398 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe37c0047a0 con 0x564ee15fcc50 2024-09-16T02:05:21.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.758+0000 7fe377400640 1 -- 172.21.15.103:0/3087083398 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c010760 con 0x564ee15fcc50 2024-09-16T02:05:21.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.759+0000 7fe393a01740 1 -- 172.21.15.103:0/3087083398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 msgr2=0x564ee1606fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:21.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.759+0000 7fe389400640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3700a3350 0x7fe3700a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x564ee15f9c40 tx=0x7fe378005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:21.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.759+0000 7fe393a01740 1 --2- 172.21.15.103:0/3087083398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1606fd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009c40 tx=0x7fe37c00e720 comp rx=0 tx=0).stop 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.759+0000 7fe393a01740 1 -- 172.21.15.103:0/3087083398 shutdown_connections 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.759+0000 7fe393a01740 1 --2- 172.21.15.103:0/3087083398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1609900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.759+0000 7fe393a01740 1 --2- 172.21.15.103:0/3087083398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1606fd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.759+0000 7fe393a01740 1 --2- 172.21.15.103:0/3087083398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fc330 0x564ee15fc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.759+0000 7fe393a01740 1 -- 172.21.15.103:0/3087083398 >> 172.21.15.103:0/3087083398 conn(0x564ee15f88c0 msgr2=0x564ee15f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.759+0000 7fe393a01740 1 -- 172.21.15.103:0/3087083398 shutdown_connections 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.759+0000 7fe393a01740 1 -- 172.21.15.103:0/3087083398 wait complete. 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.760+0000 7fe393a01740 1 -- start start 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.760+0000 7fe393a01740 1 Processor -- start 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe393a01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 0x564ee16044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe393a01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee15fcc50 0x564ee1604a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe393a01740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee1607510 0x564ee1604f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe393a01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564ee160a060 con 0x564ee15fcc50 2024-09-16T02:05:21.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe393a01740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ee14ea920 con 0x564ee1607510 2024-09-16T02:05:21.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe393a01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564ee160a1e0 con 0x564ee15fc330 2024-09-16T02:05:21.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe393a01740 10 client.0 _ll_register_callbacks cb 0x564ee150e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:21.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe389400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee15fcc50 0x564ee1604a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe389400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee15fcc50 0x564ee1604a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45064/0 (socket says 172.21.15.103:45064) 2024-09-16T02:05:21.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe389400640 1 -- 172.21.15.103:0/2265801546 learned_addr learned my addr 172.21.15.103:0/2265801546 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:21.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe389e00640 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 0x564ee16044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe389400640 1 -- 172.21.15.103:0/2265801546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 msgr2=0x564ee16044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:21.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe389400640 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 0x564ee16044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe389400640 1 -- 172.21.15.103:0/2265801546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee1607510 msgr2=0x564ee1604f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:21.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe389400640 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee1607510 0x564ee1604f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.761+0000 7fe389400640 1 -- 172.21.15.103:0/2265801546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee12d32b0 con 0x564ee15fcc50 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.762+0000 7fe389400640 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee15fcc50 0x564ee1604a20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe378010c70 tx=0x7fe378004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a87fb16a0cd71a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.762+0000 7fe36be00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.762+0000 7fe36be00640 1 -- 172.21.15.103:0/2265801546 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37802e060 con 0x564ee15fcc50 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.762+0000 7fe36be00640 1 -- 172.21.15.103:0/2265801546 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fe3780200d0 con 0x564ee15fcc50 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.762+0000 7fe393a01740 1 -- 172.21.15.103:0/2265801546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe37c0047a0 con 0x564ee15fcc50 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.762+0000 7fe36be00640 1 -- 172.21.15.103:0/2265801546 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe378019040 con 0x564ee15fcc50 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.762+0000 7fe393a01740 10 client.14994 fetch_fsmap learned FSMap version 23 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.762+0000 7fe393a01740 1 -- 172.21.15.103:0/2265801546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564ee1615dc0 con 0x564ee15fcc50 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.763+0000 7fe36be00640 1 -- 172.21.15.103:0/2265801546 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe37c0047a0 con 0x564ee15fcc50 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.763+0000 7fe36be00640 1 -- 172.21.15.103:0/2265801546 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe378029ac0 con 0x564ee15fcc50 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.763+0000 7fe393a01740 10 client.14994 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:05:21.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.763+0000 7fe393a01740 10 client.14994 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.763+0000 7fe393a01740 20 client.14994 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.763+0000 7fe393a01740 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564ee16184d0 0x564ee1619940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.763+0000 7fe393a01740 4 client.14994 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.763+0000 7fe393a01740 1 -- 172.21.15.103:0/2265801546 --> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ee156f620 con 0x564ee16184d0 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.763+0000 7fe389e00640 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564ee16184d0 0x564ee1619940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.764+0000 7fe389e00640 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564ee16184d0 0x564ee1619940 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa9fa4c9037cc34 server_cookie=78c786e883b7bcd7 in_seq=0 out_seq=0 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.764+0000 7fe36be00640 10 client.14994 ms_handle_connect on v2:172.21.15.172:6838/3474994563 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe36be00640 1 -- 172.21.15.103:0/2265801546 <== mds.2 v2:172.21.15.172:6838/3474994563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ee156f620 con 0x564ee16184d0 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe36be00640 10 client.14994 handle_command_reply: tid=0 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 client.14994 shutdown 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 -- 172.21.15.103:0/2265801546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee15fcc50 msgr2=0x564ee1604a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee15fcc50 0x564ee1604a20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe378010c70 tx=0x7fe378004730 comp rx=0 tx=0).stop 2024-09-16T02:05:21.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 -- 172.21.15.103:0/2265801546 shutdown_connections 2024-09-16T02:05:21.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564ee16184d0 0x564ee1619940 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee1607510 0x564ee1604f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee15fcc50 0x564ee1604a20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 --2- 172.21.15.103:0/2265801546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 0x564ee16044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 -- 172.21.15.103:0/2265801546 >> 172.21.15.103:0/2265801546 conn(0x564ee15f88c0 msgr2=0x564ee15fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:21.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe36be00640 0 client.14994 ms_handle_reset on v2:172.21.15.172:6838/3474994563 2024-09-16T02:05:21.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 -- 172.21.15.103:0/2265801546 shutdown_connections 2024-09-16T02:05:21.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 1 -- 172.21.15.103:0/2265801546 wait complete. 2024-09-16T02:05:21.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.765+0000 7fe393a01740 20 client.14994 trim_cache size 0 max 16384 2024-09-16T02:05:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe393a01740 1 Processor -- start 2024-09-16T02:05:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe393a01740 1 -- start start 2024-09-16T02:05:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe393a01740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fc330 0x564ee15fed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe393a01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1621c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe393a01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1622160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe393a01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564ee1617c80 con 0x564ee1607510 2024-09-16T02:05:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe393a01740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ee1618350 con 0x564ee15fc330 2024-09-16T02:05:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe393a01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564ee1618a20 con 0x564ee15fcc50 2024-09-16T02:05:21.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe38a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1622160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe389400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1621c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe389400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1621c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:35208/0 (socket says 172.21.15.103:35208) 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe38a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1622160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45080/0 (socket says 172.21.15.103:45080) 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe389400640 1 -- 172.21.15.103:0/1372845890 learned_addr learned my addr 172.21.15.103:0/1372845890 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.784+0000 7fe389e00640 1 --2- 172.21.15.103:0/1372845890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fc330 0x564ee15fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.785+0000 7fe38a800640 1 -- 172.21.15.103:0/1372845890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 msgr2=0x564ee1621c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.785+0000 7fe38a800640 1 --2- 172.21.15.103:0/1372845890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1621c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.785+0000 7fe38a800640 1 -- 172.21.15.103:0/1372845890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fc330 msgr2=0x564ee15fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.785+0000 7fe38a800640 1 --2- 172.21.15.103:0/1372845890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fc330 0x564ee15fed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.785+0000 7fe38a800640 1 -- 172.21.15.103:0/1372845890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee15fc160 con 0x564ee1607510 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.785+0000 7fe389400640 1 --2- 172.21.15.103:0/1372845890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1621c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.785+0000 7fe38a800640 1 --2- 172.21.15.103:0/1372845890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1622160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe3840ee7f0 tx=0x7fe3840b9030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a817838e9f20ac4d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.785+0000 7fe376a00640 1 -- 172.21.15.103:0/1372845890 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3840bccf0 con 0x564ee1607510 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.785+0000 7fe376a00640 1 -- 172.21.15.103:0/1372845890 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe384036030 con 0x564ee1607510 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.785+0000 7fe376a00640 1 -- 172.21.15.103:0/1372845890 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3840ee930 con 0x564ee1607510 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.786+0000 7fe393a01740 1 -- 172.21.15.103:0/1372845890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 msgr2=0x564ee1622160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.786+0000 7fe393a01740 1 --2- 172.21.15.103:0/1372845890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1622160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe3840ee7f0 tx=0x7fe3840b9030 comp rx=0 tx=0).stop 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.786+0000 7fe393a01740 1 -- 172.21.15.103:0/1372845890 shutdown_connections 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.786+0000 7fe393a01740 1 --2- 172.21.15.103:0/1372845890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1622160 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.786+0000 7fe393a01740 1 --2- 172.21.15.103:0/1372845890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fcc50 0x564ee1621c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.786+0000 7fe393a01740 1 --2- 172.21.15.103:0/1372845890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fc330 0x564ee15fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.786+0000 7fe393a01740 1 -- 172.21.15.103:0/1372845890 >> 172.21.15.103:0/1372845890 conn(0x564ee15f88c0 msgr2=0x564ee1602e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.786+0000 7fe393a01740 1 -- 172.21.15.103:0/1372845890 shutdown_connections 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.786+0000 7fe393a01740 1 -- 172.21.15.103:0/1372845890 wait complete. 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.786+0000 7fe393a01740 1 -- start start 2024-09-16T02:05:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe393a01740 1 Processor -- start 2024-09-16T02:05:21.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe393a01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 0x564ee1622420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe393a01740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fcc50 0x564ee16013b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe393a01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1601970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe393a01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564ee16197c0 con 0x564ee1607510 2024-09-16T02:05:21.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe393a01740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ee16190f0 con 0x564ee15fcc50 2024-09-16T02:05:21.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe393a01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564ee1619e90 con 0x564ee15fc330 2024-09-16T02:05:21.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe393a01740 10 client.0 _ll_register_callbacks cb 0x7fe378010820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:21.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe389e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 0x564ee1622420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe389e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 0x564ee1622420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:35220/0 (socket says 172.21.15.103:35220) 2024-09-16T02:05:21.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe38a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1601970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.787+0000 7fe389e00640 1 -- 172.21.15.103:0/464806075 learned_addr learned my addr 172.21.15.103:0/464806075 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe389400640 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fcc50 0x564ee16013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe389e00640 1 -- 172.21.15.103:0/464806075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fcc50 msgr2=0x564ee16013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe389e00640 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fcc50 0x564ee16013b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe389e00640 1 -- 172.21.15.103:0/464806075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 msgr2=0x564ee1601970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe389e00640 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1601970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe389e00640 1 -- 172.21.15.103:0/464806075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee16021a0 con 0x564ee15fc330 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe389400640 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fcc50 0x564ee16013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe389e00640 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 0x564ee1622420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009c40 tx=0x7fe37c005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc61bee2791ec3e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe36be00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe36be00640 1 -- 172.21.15.103:0/464806075 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c009e30 con 0x564ee15fc330 2024-09-16T02:05:21.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe36be00640 1 -- 172.21.15.103:0/464806075 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fe37c004040 con 0x564ee15fc330 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.788+0000 7fe393a01740 1 -- 172.21.15.103:0/464806075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe384036030 con 0x564ee15fc330 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe36be00640 1 -- 172.21.15.103:0/464806075 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe37c004240 con 0x564ee15fc330 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe393a01740 10 client.24940 fetch_fsmap learned FSMap version 23 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe393a01740 1 -- 172.21.15.103:0/464806075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564ee168b4e0 con 0x564ee15fc330 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe36be00640 1 -- 172.21.15.103:0/464806075 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe384036030 con 0x564ee15fc330 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe36be00640 1 -- 172.21.15.103:0/464806075 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe37c005160 con 0x564ee15fc330 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe393a01740 10 client.24940 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe393a01740 10 client.24940 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe393a01740 20 client.24940 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe393a01740 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564ee168fd70 0x564ee168ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe393a01740 4 client.24940 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:05:21.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.789+0000 7fe393a01740 1 -- 172.21.15.103:0/464806075 --> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] -- command(tid 0: {"prefix": "status"}) -- 0x564ee1569b30 con 0x564ee168fd70 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.790+0000 7fe389400640 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564ee168fd70 0x564ee168ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.790+0000 7fe389400640 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564ee168fd70 0x564ee168ea80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f017a62b80f71ab server_cookie=dd0736b89519efb0 in_seq=0 out_seq=0 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.790+0000 7fe36be00640 10 client.24940 ms_handle_connect on v2:172.21.15.172:6838/3474994563 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.790+0000 7fe36be00640 1 -- 172.21.15.103:0/464806075 <== mds.2 v2:172.21.15.172:6838/3474994563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564ee1569b30 con 0x564ee168fd70 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.790+0000 7fe36be00640 10 client.24940 handle_command_reply: tid=0 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.790+0000 7fe393a01740 1 client.24940 shutdown 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 1 -- 172.21.15.103:0/464806075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 msgr2=0x564ee1622420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 0x564ee1622420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009c40 tx=0x7fe37c005d30 comp rx=0 tx=0).stop 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 1 -- 172.21.15.103:0/464806075 shutdown_connections 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564ee168fd70 0x564ee168ea80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ee1607510 0x564ee1601970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ee15fcc50 0x564ee16013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 1 --2- 172.21.15.103:0/464806075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee15fc330 0x564ee1622420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe36be00640 0 client.24940 ms_handle_reset on v2:172.21.15.172:6838/3474994563 2024-09-16T02:05:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 1 -- 172.21.15.103:0/464806075 >> 172.21.15.103:0/464806075 conn(0x564ee15f88c0 msgr2=0x564ee16044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 1 -- 172.21.15.103:0/464806075 shutdown_connections 2024-09-16T02:05:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 1 -- 172.21.15.103:0/464806075 wait complete. 2024-09-16T02:05:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.791+0000 7fe393a01740 20 client.24940 trim_cache size 0 max 16384 2024-09-16T02:05:21.798 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:05:21.798 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "a20dd7a4-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:05:21.798 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-16T02:05:21.798 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24505, 2024-09-16T02:05:21.799 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-16T02:05:21.799 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-16T02:05:21.799 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-16T02:05:21.799 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 94.701982787000006, 2024-09-16T02:05:21.799 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 23, 2024-09-16T02:05:21.799 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 79, 2024-09-16T02:05:21.799 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 77, 2024-09-16T02:05:21.799 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 126.08469476400001 2024-09-16T02:05:21.799 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:05:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 -- 172.21.15.103:0/1353887913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3700a3350 msgr2=0x7fe3700a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3700a3350 0x7fe3700a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x564ee15f9c40 tx=0x7fe378005f90 comp rx=0 tx=0).stop 2024-09-16T02:05:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 -- 172.21.15.103:0/1353887913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe38c106f10 msgr2=0x7fe38c19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe38c106f10 0x7fe38c19eec0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe384031a20 tx=0x7fe384031a50 comp rx=0 tx=0).stop 2024-09-16T02:05:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 -- 172.21.15.103:0/1353887913 shutdown_connections 2024-09-16T02:05:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3700a3350 0x7fe3700a5810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe38c106f10 0x7fe38c19eec0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c102790 0x7fe38c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 --2- 172.21.15.103:0/1353887913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe38c101de0 0x7fe38c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 -- 172.21.15.103:0/1353887913 >> 172.21.15.103:0/1353887913 conn(0x7fe38c0fd8c0 msgr2=0x7fe38c0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 -- 172.21.15.103:0/1353887913 shutdown_connections 2024-09-16T02:05:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:21.792+0000 7fe391000640 1 -- 172.21.15.103:0/1353887913 wait complete. 2024-09-16T02:05:22.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:21 smithi103 ceph-mon[30398]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:05:22.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:21 smithi136 ceph-mon[35323]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:05:22.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:21 smithi172 ceph-mon[35340]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:05:22.583 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T02:05:22.954 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:23.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.493+0000 7fe360a00640 1 -- 172.21.15.103:0/1739416133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe35c102360 msgr2=0x7fe35c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:23.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.493+0000 7fe360a00640 1 --2- 172.21.15.103:0/1739416133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe35c102360 0x7fe35c10e890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe354009cd0 tx=0x7fe35402a8e0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.493+0000 7fe360a00640 1 -- 172.21.15.103:0/1739416133 shutdown_connections 2024-09-16T02:05:23.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.493+0000 7fe360a00640 1 --2- 172.21.15.103:0/1739416133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe35c102360 0x7fe35c10e890 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.493+0000 7fe360a00640 1 --2- 172.21.15.103:0/1739416133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c1019c0 0x7fe35c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.493+0000 7fe360a00640 1 --2- 172.21.15.103:0/1739416133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe35c1079c0 0x7fe35c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.493+0000 7fe360a00640 1 -- 172.21.15.103:0/1739416133 >> 172.21.15.103:0/1739416133 conn(0x7fe35c0fd740 msgr2=0x7fe35c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:23.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.493+0000 7fe360a00640 1 -- 172.21.15.103:0/1739416133 shutdown_connections 2024-09-16T02:05:23.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.493+0000 7fe360a00640 1 -- 172.21.15.103:0/1739416133 wait complete. 2024-09-16T02:05:23.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.494+0000 7fe360a00640 1 Processor -- start 2024-09-16T02:05:23.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.494+0000 7fe360a00640 1 -- start start 2024-09-16T02:05:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.495+0000 7fe360a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c1019c0 0x7fe35c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.495+0000 7fe360a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe35c102360 0x7fe35c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.495+0000 7fe360a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe35c1079c0 0x7fe35c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.495+0000 7fe360a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c111e10 con 0x7fe35c1079c0 2024-09-16T02:05:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.495+0000 7fe360a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c111c90 con 0x7fe35c102360 2024-09-16T02:05:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.495+0000 7fe360a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c111f90 con 0x7fe35c1019c0 2024-09-16T02:05:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.495+0000 7fe35b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe35c1079c0 0x7fe35c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.495+0000 7fe35b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe35c1079c0 0x7fe35c19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45096/0 (socket says 172.21.15.103:45096) 2024-09-16T02:05:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.495+0000 7fe35b200640 1 -- 172.21.15.103:0/2304404231 learned_addr learned my addr 172.21.15.103:0/2304404231 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.495+0000 7fe359e00640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe35c102360 0x7fe35c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe35b200640 1 -- 172.21.15.103:0/2304404231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c1019c0 msgr2=0x7fe35c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe35b200640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c1019c0 0x7fe35c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe35b200640 1 -- 172.21.15.103:0/2304404231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe35c102360 msgr2=0x7fe35c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe35b200640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe35c102360 0x7fe35c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe35b200640 1 -- 172.21.15.103:0/2304404231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c19f490 con 0x7fe35c1079c0 2024-09-16T02:05:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe359e00640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe35c102360 0x7fe35c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe35b200640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe35c1079c0 0x7fe35c19edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe354033040 tx=0x7fe354003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe353400640 1 -- 172.21.15.103:0/2304404231 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354004040 con 0x7fe35c1079c0 2024-09-16T02:05:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe360a00640 1 -- 172.21.15.103:0/2304404231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35c19f720 con 0x7fe35c1079c0 2024-09-16T02:05:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe353400640 1 -- 172.21.15.103:0/2304404231 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3540046c0 con 0x7fe35c1079c0 2024-09-16T02:05:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.496+0000 7fe353400640 1 -- 172.21.15.103:0/2304404231 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354046480 con 0x7fe35c1079c0 2024-09-16T02:05:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.497+0000 7fe360a00640 1 -- 172.21.15.103:0/2304404231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35c1a7060 con 0x7fe35c1079c0 2024-09-16T02:05:23.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.499+0000 7fe353400640 1 -- 172.21.15.103:0/2304404231 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe3540305a0 con 0x7fe35c1079c0 2024-09-16T02:05:23.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe363ff2740 1 Processor -- start 2024-09-16T02:05:23.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe363ff2740 1 -- start start 2024-09-16T02:05:23.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 0x55af1f8358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f840180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8406c0 0x55af1f842ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe363ff2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55af1f79b0b0 con 0x55af1f835e00 2024-09-16T02:05:23.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe363ff2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55af1f742ab0 con 0x55af1f8354e0 2024-09-16T02:05:23.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe363ff2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55af1f797070 con 0x55af1f8406c0 2024-09-16T02:05:23.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe35a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f840180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe35a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f840180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45110/0 (socket says 172.21.15.103:45110) 2024-09-16T02:05:23.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe35a800640 1 -- 172.21.15.103:0/1742774089 learned_addr learned my addr 172.21.15.103:0/1742774089 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:23.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.500+0000 7fe359e00640 1 --2- 172.21.15.103:0/1742774089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 0x55af1f8358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.501+0000 7fe359e00640 1 --2- 172.21.15.103:0/1742774089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8406c0 0x55af1f842ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.501+0000 7fe35a800640 1 -- 172.21.15.103:0/1742774089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8406c0 msgr2=0x55af1f842ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:23.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.501+0000 7fe35a800640 1 --2- 172.21.15.103:0/1742774089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8406c0 0x55af1f842ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.501+0000 7fe35a800640 1 -- 172.21.15.103:0/1742774089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 msgr2=0x55af1f8358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:23.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.501+0000 7fe35a800640 1 --2- 172.21.15.103:0/1742774089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 0x55af1f8358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.501+0000 7fe35a800640 1 -- 172.21.15.103:0/1742774089 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1f74f4a0 con 0x55af1f835e00 2024-09-16T02:05:23.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.501+0000 7fe353400640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3400a32d0 0x7fe3400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.501+0000 7fe35a800640 1 --2- 172.21.15.103:0/1742774089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f840180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00bad0 tx=0x7fe34c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333ad4c469e2c626 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.501+0000 7fe353400640 1 -- 172.21.15.103:0/2304404231 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fe35402e030 con 0x7fe35c1079c0 2024-09-16T02:05:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.501+0000 7fe359e00640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3400a32d0 0x7fe3400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.502+0000 7fe347400640 1 -- 172.21.15.103:0/1742774089 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c016070 con 0x55af1f835e00 2024-09-16T02:05:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.502+0000 7fe347400640 1 -- 172.21.15.103:0/1742774089 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe34c00dca0 con 0x55af1f835e00 2024-09-16T02:05:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.502+0000 7fe347400640 1 -- 172.21.15.103:0/1742774089 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c004a00 con 0x55af1f835e00 2024-09-16T02:05:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.502+0000 7fe363ff2740 1 -- 172.21.15.103:0/1742774089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 msgr2=0x55af1f840180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:23.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.502+0000 7fe363ff2740 1 --2- 172.21.15.103:0/1742774089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f840180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00bad0 tx=0x7fe34c00bf90 comp rx=0 tx=0).stop 2024-09-16T02:05:23.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.502+0000 7fe359e00640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3400a32d0 0x7fe3400a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55af1f832df0 tx=0x7fe34800c510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:23.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.503+0000 7fe363ff2740 1 -- 172.21.15.103:0/1742774089 shutdown_connections 2024-09-16T02:05:23.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.503+0000 7fe363ff2740 1 --2- 172.21.15.103:0/1742774089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8406c0 0x55af1f842ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.503+0000 7fe363ff2740 1 --2- 172.21.15.103:0/1742774089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f840180 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.503+0000 7fe363ff2740 1 --2- 172.21.15.103:0/1742774089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 0x55af1f8358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.503+0000 7fe363ff2740 1 -- 172.21.15.103:0/1742774089 >> 172.21.15.103:0/1742774089 conn(0x55af1f831a70 msgr2=0x55af1f831e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:23.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.503+0000 7fe363ff2740 1 -- 172.21.15.103:0/1742774089 shutdown_connections 2024-09-16T02:05:23.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.503+0000 7fe363ff2740 1 -- 172.21.15.103:0/1742774089 wait complete. 2024-09-16T02:05:23.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.503+0000 7fe363ff2740 1 -- start start 2024-09-16T02:05:23.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe363ff2740 1 Processor -- start 2024-09-16T02:05:23.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 0x55af1f83d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f835e00 0x55af1f83db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f8406c0 0x55af1f83e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe363ff2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55af1f843330 con 0x55af1f8406c0 2024-09-16T02:05:23.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe363ff2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55af1f723920 con 0x55af1f8354e0 2024-09-16T02:05:23.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe363ff2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55af1f8434b0 con 0x55af1f835e00 2024-09-16T02:05:23.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe363ff2740 10 client.0 _ll_register_callbacks cb 0x55af1f7478a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:23.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe35b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f8406c0 0x55af1f83e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe35a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 0x55af1f83d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe35b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f8406c0 0x55af1f83e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45118/0 (socket says 172.21.15.103:45118) 2024-09-16T02:05:23.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe35b200640 1 -- 172.21.15.103:0/3121423335 learned_addr learned my addr 172.21.15.103:0/3121423335 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:23.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.504+0000 7fe359e00640 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f835e00 0x55af1f83db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe35b200640 1 -- 172.21.15.103:0/3121423335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f835e00 msgr2=0x55af1f83db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:23.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe35b200640 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f835e00 0x55af1f83db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe35b200640 1 -- 172.21.15.103:0/3121423335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 msgr2=0x55af1f83d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:23.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe35b200640 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 0x55af1f83d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe35b200640 1 -- 172.21.15.103:0/3121423335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1f50c2b0 con 0x55af1f8406c0 2024-09-16T02:05:23.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe35a800640 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 0x55af1f83d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:23.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe359e00640 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f835e00 0x55af1f83db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:23.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe35b200640 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f8406c0 0x55af1f83e040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3540c5cd0 tx=0x7fe3540bd0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e5b202267d28da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:23.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe33be00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:05:23.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe33be00640 1 -- 172.21.15.103:0/3121423335 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3540d5070 con 0x55af1f8406c0 2024-09-16T02:05:23.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe33be00640 1 -- 172.21.15.103:0/3121423335 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fe3540d6260 con 0x55af1f8406c0 2024-09-16T02:05:23.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.505+0000 7fe363ff2740 1 -- 172.21.15.103:0/3121423335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe34c00dca0 con 0x55af1f8406c0 2024-09-16T02:05:23.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe33be00640 1 -- 172.21.15.103:0/3121423335 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe3540bd590 con 0x55af1f8406c0 2024-09-16T02:05:23.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe363ff2740 10 client.15012 fetch_fsmap learned FSMap version 23 2024-09-16T02:05:23.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe363ff2740 1 -- 172.21.15.103:0/3121423335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55af1f84eff0 con 0x55af1f8406c0 2024-09-16T02:05:23.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe33be00640 1 -- 172.21.15.103:0/3121423335 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe34c00dca0 con 0x55af1f8406c0 2024-09-16T02:05:23.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe33be00640 1 -- 172.21.15.103:0/3121423335 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe354030070 con 0x55af1f8406c0 2024-09-16T02:05:23.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe363ff2740 10 client.15012 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:05:23.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe363ff2740 10 client.15012 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T02:05:23.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe363ff2740 20 client.15012 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe363ff2740 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55af1f8516d0 0x55af1f852b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe363ff2740 4 client.15012 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.506+0000 7fe363ff2740 1 -- 172.21.15.103:0/3121423335 --> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af1f7a8620 con 0x55af1f8516d0 2024-09-16T02:05:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.507+0000 7fe35a800640 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55af1f8516d0 0x55af1f852b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.507+0000 7fe35a800640 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55af1f8516d0 0x55af1f852b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80ff062549df058e server_cookie=a388f129590065c9 in_seq=0 out_seq=0 2024-09-16T02:05:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.507+0000 7fe33be00640 10 client.15012 ms_handle_connect on v2:172.21.15.136:6836/236571207 2024-09-16T02:05:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe33be00640 1 -- 172.21.15.103:0/3121423335 <== mds.1 v2:172.21.15.136:6836/236571207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af1f7a8620 con 0x55af1f8516d0 2024-09-16T02:05:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe33be00640 10 client.15012 handle_command_reply: tid=0 2024-09-16T02:05:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe363ff2740 1 client.15012 shutdown 2024-09-16T02:05:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe363ff2740 1 -- 172.21.15.103:0/3121423335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f8406c0 msgr2=0x55af1f83e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe363ff2740 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f8406c0 0x55af1f83e040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3540c5cd0 tx=0x7fe3540bd0d0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe363ff2740 1 -- 172.21.15.103:0/3121423335 shutdown_connections 2024-09-16T02:05:23.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe363ff2740 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f8406c0 0x55af1f83e040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe363ff2740 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f835e00 0x55af1f83db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe363ff2740 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55af1f8516d0 0x55af1f852b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe363ff2740 1 --2- 172.21.15.103:0/3121423335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8354e0 0x55af1f83d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe363ff2740 1 -- 172.21.15.103:0/3121423335 >> 172.21.15.103:0/3121423335 conn(0x55af1f831a70 msgr2=0x55af1f830bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:23.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe33be00640 0 client.15012 ms_handle_reset on v2:172.21.15.136:6836/236571207 2024-09-16T02:05:23.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.508+0000 7fe363ff2740 1 -- 172.21.15.103:0/3121423335 shutdown_connections 2024-09-16T02:05:23.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.509+0000 7fe363ff2740 1 -- 172.21.15.103:0/3121423335 wait complete. 2024-09-16T02:05:23.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.509+0000 7fe363ff2740 20 client.15012 trim_cache size 0 max 16384 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.527+0000 7fe363ff2740 1 Processor -- start 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.527+0000 7fe363ff2740 1 -- start start 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.527+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 0x55af1f852470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.527+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f837f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.527+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 0x55af1f83be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.527+0000 7fe363ff2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55af1f850e80 con 0x55af1f835e00 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.527+0000 7fe363ff2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55af1f851550 con 0x55af1f8406c0 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.527+0000 7fe363ff2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55af1f851c20 con 0x55af1f8354e0 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe359e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f837f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe359e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f837f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45120/0 (socket says 172.21.15.103:45120) 2024-09-16T02:05:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe359e00640 1 -- 172.21.15.103:0/892688139 learned_addr learned my addr 172.21.15.103:0/892688139 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe35a800640 1 --2- 172.21.15.103:0/892688139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 0x55af1f852470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe35b200640 1 --2- 172.21.15.103:0/892688139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 0x55af1f83be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe35a800640 1 -- 172.21.15.103:0/892688139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 msgr2=0x55af1f83be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe35a800640 1 --2- 172.21.15.103:0/892688139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 0x55af1f83be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe35a800640 1 -- 172.21.15.103:0/892688139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 msgr2=0x55af1f837f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe35a800640 1 --2- 172.21.15.103:0/892688139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f837f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe35a800640 1 -- 172.21.15.103:0/892688139 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1f835310 con 0x55af1f8354e0 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe359e00640 1 --2- 172.21.15.103:0/892688139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f837f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.528+0000 7fe35a800640 1 --2- 172.21.15.103:0/892688139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 0x55af1f852470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe34c004700 tx=0x7fe34c005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90c33209bb3237c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe346a00640 1 -- 172.21.15.103:0/892688139 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c016040 con 0x55af1f8354e0 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe346a00640 1 -- 172.21.15.103:0/892688139 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe34c00b5e0 con 0x55af1f8354e0 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe346a00640 1 -- 172.21.15.103:0/892688139 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0038e0 con 0x55af1f8354e0 2024-09-16T02:05:23.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe363ff2740 1 -- 172.21.15.103:0/892688139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 msgr2=0x55af1f852470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe363ff2740 1 --2- 172.21.15.103:0/892688139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 0x55af1f852470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe34c004700 tx=0x7fe34c005420 comp rx=0 tx=0).stop 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe363ff2740 1 -- 172.21.15.103:0/892688139 shutdown_connections 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe363ff2740 1 --2- 172.21.15.103:0/892688139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 0x55af1f83be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe363ff2740 1 --2- 172.21.15.103:0/892688139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f837f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe363ff2740 1 --2- 172.21.15.103:0/892688139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 0x55af1f852470 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe363ff2740 1 -- 172.21.15.103:0/892688139 >> 172.21.15.103:0/892688139 conn(0x55af1f831a70 msgr2=0x55af1f850340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe363ff2740 1 -- 172.21.15.103:0/892688139 shutdown_connections 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.529+0000 7fe363ff2740 1 -- 172.21.15.103:0/892688139 wait complete. 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.530+0000 7fe363ff2740 1 -- start start 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.530+0000 7fe363ff2740 1 Processor -- start 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.530+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 0x55af1f85b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.530+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f85b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.530+0000 7fe363ff2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 0x55af1f8543e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.530+0000 7fe363ff2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55af1f8529c0 con 0x55af1f835e00 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.530+0000 7fe363ff2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55af1f8522f0 con 0x55af1f8406c0 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.530+0000 7fe363ff2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55af1f853090 con 0x55af1f8354e0 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe363ff2740 10 client.0 _ll_register_callbacks cb 0x7fe3540b9830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe359e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f85b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe359e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f85b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45124/0 (socket says 172.21.15.103:45124) 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe359e00640 1 -- 172.21.15.103:0/714524481 learned_addr learned my addr 172.21.15.103:0/714524481 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe35a800640 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 0x55af1f85b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe35b200640 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 0x55af1f8543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe35a800640 1 -- 172.21.15.103:0/714524481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 msgr2=0x55af1f8543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe35a800640 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 0x55af1f8543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe35a800640 1 -- 172.21.15.103:0/714524481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 msgr2=0x55af1f85b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe35a800640 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f85b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe35a800640 1 -- 172.21.15.103:0/714524481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1f832700 con 0x55af1f8354e0 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe35b200640 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 0x55af1f8543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.531+0000 7fe359e00640 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f85b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.532+0000 7fe35a800640 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 0x55af1f85b020 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00b9f0 tx=0x7fe34c00d8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caf3c230e6d895d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.532+0000 7fe33be00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.532+0000 7fe33be00640 1 -- 172.21.15.103:0/714524481 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c016040 con 0x55af1f8354e0 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.532+0000 7fe363ff2740 1 -- 172.21.15.103:0/714524481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe34c00b5e0 con 0x55af1f8354e0 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.532+0000 7fe33be00640 1 -- 172.21.15.103:0/714524481 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7fe34c00dca0 con 0x55af1f8354e0 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.532+0000 7fe33be00640 1 -- 172.21.15.103:0/714524481 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe34c00dea0 con 0x55af1f8354e0 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.532+0000 7fe363ff2740 10 client.24955 fetch_fsmap learned FSMap version 23 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.532+0000 7fe363ff2740 1 -- 172.21.15.103:0/714524481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55af1f8594d0 con 0x55af1f8354e0 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.533+0000 7fe33be00640 1 -- 172.21.15.103:0/714524481 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe34c00b5e0 con 0x55af1f8354e0 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.533+0000 7fe33be00640 1 -- 172.21.15.103:0/714524481 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe34c024730 con 0x55af1f8354e0 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.533+0000 7fe363ff2740 10 client.24955 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:05:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.533+0000 7fe363ff2740 10 client.24955 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.533+0000 7fe363ff2740 20 client.24955 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.533+0000 7fe363ff2740 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55af1f8bfc90 0x55af1f8be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.533+0000 7fe363ff2740 4 client.24955 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.533+0000 7fe363ff2740 1 -- 172.21.15.103:0/714524481 --> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] -- command(tid 0: {"prefix": "status"}) -- 0x55af1f7a2b30 con 0x55af1f8bfc90 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.533+0000 7fe359e00640 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55af1f8bfc90 0x55af1f8be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe359e00640 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55af1f8bfc90 0x55af1f8be9a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84d693d533af0e5f server_cookie=24aee8d99bb7444c in_seq=0 out_seq=0 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe33be00640 10 client.24955 ms_handle_connect on v2:172.21.15.136:6836/236571207 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe33be00640 1 -- 172.21.15.103:0/714524481 <== mds.1 v2:172.21.15.136:6836/236571207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55af1f7a2b30 con 0x55af1f8bfc90 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe33be00640 10 client.24955 handle_command_reply: tid=0 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe363ff2740 1 client.24955 shutdown 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe363ff2740 1 -- 172.21.15.103:0/714524481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 msgr2=0x55af1f85b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe363ff2740 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 0x55af1f85b020 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00b9f0 tx=0x7fe34c00d8b0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe363ff2740 1 -- 172.21.15.103:0/714524481 shutdown_connections 2024-09-16T02:05:23.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe363ff2740 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55af1f8bfc90 0x55af1f8be9a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe363ff2740 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55af1f8406c0 0x55af1f8543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe363ff2740 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af1f835e00 0x55af1f85b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe33be00640 0 client.24955 ms_handle_reset on v2:172.21.15.136:6836/236571207 2024-09-16T02:05:23.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe363ff2740 1 --2- 172.21.15.103:0/714524481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af1f8354e0 0x55af1f85b020 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.534+0000 7fe363ff2740 1 -- 172.21.15.103:0/714524481 >> 172.21.15.103:0/714524481 conn(0x55af1f831a70 msgr2=0x55af1f83bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:23.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.535+0000 7fe363ff2740 1 -- 172.21.15.103:0/714524481 shutdown_connections 2024-09-16T02:05:23.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.535+0000 7fe363ff2740 1 -- 172.21.15.103:0/714524481 wait complete. 2024-09-16T02:05:23.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.535+0000 7fe363ff2740 20 client.24955 trim_cache size 0 max 16384 2024-09-16T02:05:23.548 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:05:23.548 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "a20dd7a4-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24506, 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 97.454236086999998, 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 23, 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 79, 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 77, 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 130.21319212899999 2024-09-16T02:05:23.549 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:05:23.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 -- 172.21.15.103:0/2304404231 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3400a32d0 msgr2=0x7fe3400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:23.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3400a32d0 0x7fe3400a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55af1f832df0 tx=0x7fe34800c510 comp rx=0 tx=0).stop 2024-09-16T02:05:23.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 -- 172.21.15.103:0/2304404231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe35c1079c0 msgr2=0x7fe35c19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:23.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe35c1079c0 0x7fe35c19edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe354033040 tx=0x7fe354003970 comp rx=0 tx=0).stop 2024-09-16T02:05:23.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 -- 172.21.15.103:0/2304404231 shutdown_connections 2024-09-16T02:05:23.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe3400a32d0 0x7fe3400a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe35c1079c0 0x7fe35c19edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe35c102360 0x7fe35c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 --2- 172.21.15.103:0/2304404231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c1019c0 0x7fe35c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:23.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 -- 172.21.15.103:0/2304404231 >> 172.21.15.103:0/2304404231 conn(0x7fe35c0fd740 msgr2=0x7fe35c10d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:23.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 -- 172.21.15.103:0/2304404231 shutdown_connections 2024-09-16T02:05:23.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:23.536+0000 7fe360a00640 1 -- 172.21.15.103:0/2304404231 wait complete. 2024-09-16T02:05:23.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:23 smithi103 ceph-mon[30398]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:23 smithi136 ceph-mon[35323]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:23 smithi172 ceph-mon[35340]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:24.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T02:05:24.585 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:25.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:24 smithi103 ceph-mon[30398]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:25.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:24 smithi136 ceph-mon[35323]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:25.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:24 smithi172 ceph-mon[35340]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:25.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.144+0000 7f42c6800640 1 -- 172.21.15.103:0/3781933358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c0101de0 msgr2=0x7f42c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.144+0000 7f42c6800640 1 --2- 172.21.15.103:0/3781933358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c0101de0 0x7f42c01021c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009900 tx=0x7f42b002a410 comp rx=0 tx=0).stop 2024-09-16T02:05:25.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.144+0000 7f42c6800640 1 -- 172.21.15.103:0/3781933358 shutdown_connections 2024-09-16T02:05:25.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.144+0000 7f42c6800640 1 --2- 172.21.15.103:0/3781933358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c0106f10 0x7f42c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.144+0000 7f42c6800640 1 --2- 172.21.15.103:0/3781933358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c0102790 0x7f42c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.144+0000 7f42c6800640 1 --2- 172.21.15.103:0/3781933358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c0101de0 0x7f42c01021c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.144+0000 7f42c6800640 1 -- 172.21.15.103:0/3781933358 >> 172.21.15.103:0/3781933358 conn(0x7f42c00fd8c0 msgr2=0x7f42c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.145+0000 7f42c6800640 1 -- 172.21.15.103:0/3781933358 shutdown_connections 2024-09-16T02:05:25.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.145+0000 7f42c6800640 1 -- 172.21.15.103:0/3781933358 wait complete. 2024-09-16T02:05:25.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42c6800640 1 Processor -- start 2024-09-16T02:05:25.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42c6800640 1 -- start start 2024-09-16T02:05:25.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42c6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c0101de0 0x7f42c019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42c6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c0102790 0x7f42c019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42c6800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c0106f10 0x7f42c019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42c6800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0111f70 con 0x7f42c0102790 2024-09-16T02:05:25.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42c6800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0111df0 con 0x7f42c0101de0 2024-09-16T02:05:25.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42c6800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f42c01120f0 con 0x7f42c0106f10 2024-09-16T02:05:25.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42bf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c0101de0 0x7f42c019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42bf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c0101de0 0x7f42c019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:57132/0 (socket says 172.21.15.103:57132) 2024-09-16T02:05:25.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42bf400640 1 -- 172.21.15.103:0/3010876853 learned_addr learned my addr 172.21.15.103:0/3010876853 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:25.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42bea00640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c0102790 0x7f42c019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.146+0000 7f42bfe00640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c0106f10 0x7f42c019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.147+0000 7f42bf400640 1 -- 172.21.15.103:0/3010876853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c0106f10 msgr2=0x7f42c019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.147+0000 7f42bf400640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c0106f10 0x7f42c019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.147+0000 7f42bf400640 1 -- 172.21.15.103:0/3010876853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c0102790 msgr2=0x7f42c019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.147+0000 7f42bf400640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c0102790 0x7f42c019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.147+0000 7f42bf400640 1 -- 172.21.15.103:0/3010876853 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c019f620 con 0x7f42c0101de0 2024-09-16T02:05:25.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.147+0000 7f42bf400640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c0101de0 0x7f42c019a670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42b0006010 tx=0x7f42b0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.147+0000 7f42b7e00640 1 -- 172.21.15.103:0/3010876853 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b003f030 con 0x7f42c0101de0 2024-09-16T02:05:25.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.148+0000 7f42c6800640 1 -- 172.21.15.103:0/3010876853 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c019f8b0 con 0x7f42c0101de0 2024-09-16T02:05:25.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.148+0000 7f42b7e00640 1 -- 172.21.15.103:0/3010876853 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42b0004440 con 0x7f42c0101de0 2024-09-16T02:05:25.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.148+0000 7f42b7e00640 1 -- 172.21.15.103:0/3010876853 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0035070 con 0x7f42c0101de0 2024-09-16T02:05:25.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.148+0000 7f42c6800640 1 -- 172.21.15.103:0/3010876853 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c01a71f0 con 0x7f42c0101de0 2024-09-16T02:05:25.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.150+0000 7f42c93c3740 1 Processor -- start 2024-09-16T02:05:25.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42b7e00640 1 -- 172.21.15.103:0/3010876853 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f42b0042030 con 0x7f42c0101de0 2024-09-16T02:05:25.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42c93c3740 1 -- start start 2024-09-16T02:05:25.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652c330 0x55b7a652c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 0x55b7a6536fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a6539900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42c93c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b7a64920b0 con 0x55b7a6537510 2024-09-16T02:05:25.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42c93c3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b7a6439ab0 con 0x55b7a652c330 2024-09-16T02:05:25.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42c93c3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b7a648e070 con 0x55b7a652cc50 2024-09-16T02:05:25.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42bea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a6539900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42bea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a6539900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45152/0 (socket says 172.21.15.103:45152) 2024-09-16T02:05:25.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42bea00640 1 -- 172.21.15.103:0/519314463 learned_addr learned my addr 172.21.15.103:0/519314463 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:25.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42bfe00640 1 --2- 172.21.15.103:0/519314463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652c330 0x55b7a652c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.151+0000 7f42bf400640 1 --2- 172.21.15.103:0/519314463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 0x55b7a6536fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.152+0000 7f42bfe00640 1 -- 172.21.15.103:0/519314463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 msgr2=0x55b7a6536fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.152+0000 7f42bfe00640 1 --2- 172.21.15.103:0/519314463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 0x55b7a6536fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.152+0000 7f42bfe00640 1 -- 172.21.15.103:0/519314463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 msgr2=0x55b7a6539900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.152+0000 7f42bfe00640 1 --2- 172.21.15.103:0/519314463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a6539900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.152+0000 7f42bfe00640 1 -- 172.21.15.103:0/519314463 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a64464a0 con 0x55b7a652c330 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.152+0000 7f42bf400640 1 --2- 172.21.15.103:0/519314463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 0x55b7a6536fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.152+0000 7f42bfe00640 1 --2- 172.21.15.103:0/519314463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652c330 0x55b7a652c710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009f00 tx=0x7f42b8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25efdd57b59a6c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.152+0000 7f42b4c00640 1 -- 172.21.15.103:0/519314463 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8004d20 con 0x55b7a652c330 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42b4c00640 1 -- 172.21.15.103:0/519314463 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42b8004050 con 0x55b7a652c330 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42b4c00640 1 -- 172.21.15.103:0/519314463 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8005280 con 0x55b7a652c330 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42b7e00640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f42a40a3350 0x7f42a40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42b7e00640 1 -- 172.21.15.103:0/3010876853 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f42b00efde0 con 0x7f42c0101de0 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42bf400640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f42a40a3350 0x7f42a40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42c93c3740 1 -- 172.21.15.103:0/519314463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652c330 msgr2=0x55b7a652c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42c93c3740 1 --2- 172.21.15.103:0/519314463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652c330 0x55b7a652c710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009f00 tx=0x7f42b8011810 comp rx=0 tx=0).stop 2024-09-16T02:05:25.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42c93c3740 1 -- 172.21.15.103:0/519314463 shutdown_connections 2024-09-16T02:05:25.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42c93c3740 1 --2- 172.21.15.103:0/519314463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a6539900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42c93c3740 1 --2- 172.21.15.103:0/519314463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 0x55b7a6536fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42c93c3740 1 --2- 172.21.15.103:0/519314463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652c330 0x55b7a652c710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.153+0000 7f42c93c3740 1 -- 172.21.15.103:0/519314463 >> 172.21.15.103:0/519314463 conn(0x55b7a65288c0 msgr2=0x55b7a6528ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.154+0000 7f42bf400640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f42a40a3350 0x7f42a40a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f42b00ed750 tx=0x7f42b002a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.154+0000 7f42c93c3740 1 -- 172.21.15.103:0/519314463 shutdown_connections 2024-09-16T02:05:25.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.154+0000 7f42c93c3740 1 -- 172.21.15.103:0/519314463 wait complete. 2024-09-16T02:05:25.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.154+0000 7f42c93c3740 1 -- start start 2024-09-16T02:05:25.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.154+0000 7f42c93c3740 1 Processor -- start 2024-09-16T02:05:25.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a652c330 0x55b7a6534580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 0x55b7a6534ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a6537510 0x55b7a6535000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42c93c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b7a653a060 con 0x55b7a652c330 2024-09-16T02:05:25.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42c93c3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b7a641a920 con 0x55b7a6537510 2024-09-16T02:05:25.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42c93c3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b7a653a1e0 con 0x55b7a652cc50 2024-09-16T02:05:25.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42c93c3740 10 client.0 _ll_register_callbacks cb 0x55b7a643e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a652c330 0x55b7a6534580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a652c330 0x55b7a6534580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45164/0 (socket says 172.21.15.103:45164) 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42bfe00640 1 -- 172.21.15.103:0/1201350327 learned_addr learned my addr 172.21.15.103:0/1201350327 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42bea00640 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 0x55b7a6534ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42bfe00640 1 -- 172.21.15.103:0/1201350327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 msgr2=0x55b7a6534ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42bfe00640 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 0x55b7a6534ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42bfe00640 1 -- 172.21.15.103:0/1201350327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a6537510 msgr2=0x55b7a6535000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42bfe00640 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a6537510 0x55b7a6535000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.155+0000 7f42bfe00640 1 -- 172.21.15.103:0/1201350327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a62032b0 con 0x55b7a652c330 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.156+0000 7f42bfe00640 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a652c330 0x55b7a6534580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f42b801b040 tx=0x7f42b8009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc43ca9f3c4b50f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.156+0000 7f42a9600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.156+0000 7f42a9600640 1 -- 172.21.15.103:0/1201350327 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8029070 con 0x55b7a652c330 2024-09-16T02:05:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.156+0000 7f42a9600640 1 -- 172.21.15.103:0/1201350327 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f42b8025460 con 0x55b7a652c330 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.156+0000 7f42c93c3740 1 -- 172.21.15.103:0/1201350327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b8004050 con 0x55b7a652c330 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.156+0000 7f42a9600640 1 -- 172.21.15.103:0/1201350327 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f42b8035050 con 0x55b7a652c330 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.156+0000 7f42c93c3740 10 client.15030 fetch_fsmap learned FSMap version 23 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.156+0000 7f42c93c3740 1 -- 172.21.15.103:0/1201350327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b7a6545e10 con 0x55b7a652c330 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.157+0000 7f42a9600640 1 -- 172.21.15.103:0/1201350327 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42b8004050 con 0x55b7a652c330 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.157+0000 7f42a9600640 1 -- 172.21.15.103:0/1201350327 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f42b8019760 con 0x55b7a652c330 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.157+0000 7f42c93c3740 10 client.15030 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.157+0000 7f42c93c3740 10 client.15030 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.157+0000 7f42c93c3740 20 client.15030 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.157+0000 7f42c93c3740 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55b7a6548210 0x55b7a6549680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.157+0000 7f42c93c3740 4 client.15030 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.157+0000 7f42c93c3740 1 -- 172.21.15.103:0/1201350327 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7a649f620 con 0x55b7a6548210 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.158+0000 7f42bea00640 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55b7a6548210 0x55b7a6549680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.158+0000 7f42bea00640 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55b7a6548210 0x55b7a6549680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44022e5dc65c8be2 server_cookie=4ddb8871d8491d6a in_seq=0 out_seq=0 2024-09-16T02:05:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.158+0000 7f42a9600640 10 client.15030 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42a9600640 1 -- 172.21.15.103:0/1201350327 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7a649f620 con 0x55b7a6548210 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42a9600640 10 client.15030 handle_command_reply: tid=0 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 client.15030 shutdown 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 -- 172.21.15.103:0/1201350327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a652c330 msgr2=0x55b7a6534580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a652c330 0x55b7a6534580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f42b801b040 tx=0x7f42b8009cf0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 -- 172.21.15.103:0/1201350327 shutdown_connections 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55b7a6548210 0x55b7a6549680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a6537510 0x55b7a6535000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652cc50 0x55b7a6534ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 --2- 172.21.15.103:0/1201350327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a652c330 0x55b7a6534580 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42a9600640 0 client.15030 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 -- 172.21.15.103:0/1201350327 >> 172.21.15.103:0/1201350327 conn(0x55b7a65288c0 msgr2=0x55b7a652e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 -- 172.21.15.103:0/1201350327 shutdown_connections 2024-09-16T02:05:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.159+0000 7f42c93c3740 1 -- 172.21.15.103:0/1201350327 wait complete. 2024-09-16T02:05:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.160+0000 7f42c93c3740 20 client.15030 trim_cache size 0 max 16384 2024-09-16T02:05:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.177+0000 7f42c93c3740 1 Processor -- start 2024-09-16T02:05:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42c93c3740 1 -- start start 2024-09-16T02:05:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 0x55b7a65488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 0x55b7a6528210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42c93c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b7a65479c0 con 0x55b7a6537510 2024-09-16T02:05:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42c93c3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b7a6548090 con 0x55b7a652cc50 2024-09-16T02:05:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42c93c3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b7a6548760 con 0x55b7a652c330 2024-09-16T02:05:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42bea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 0x55b7a6528210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45168/0 (socket says 172.21.15.103:45168) 2024-09-16T02:05:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42bfe00640 1 -- 172.21.15.103:0/839092482 learned_addr learned my addr 172.21.15.103:0/839092482 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.178+0000 7f42bfe00640 1 --2- 172.21.15.103:0/839092482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 0x55b7a65488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.179+0000 7f42bea00640 1 -- 172.21.15.103:0/839092482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 msgr2=0x55b7a65488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.179+0000 7f42bea00640 1 --2- 172.21.15.103:0/839092482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 0x55b7a65488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.179+0000 7f42bea00640 1 -- 172.21.15.103:0/839092482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 msgr2=0x55b7a65335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.179+0000 7f42bea00640 1 --2- 172.21.15.103:0/839092482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.179+0000 7f42bea00640 1 -- 172.21.15.103:0/839092482 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a652ca40 con 0x55b7a652cc50 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.179+0000 7f42bfe00640 1 --2- 172.21.15.103:0/839092482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.179+0000 7f42bea00640 1 --2- 172.21.15.103:0/839092482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 0x55b7a6528210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55b7a6535bc0 tx=0x7f42ac005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=102ce9c9a89e02fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.179+0000 7f42abe00640 1 -- 172.21.15.103:0/839092482 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0042f0 con 0x55b7a652cc50 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.179+0000 7f42abe00640 1 -- 172.21.15.103:0/839092482 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42ac004490 con 0x55b7a652cc50 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.179+0000 7f42abe00640 1 -- 172.21.15.103:0/839092482 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0035f0 con 0x55b7a652cc50 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 -- 172.21.15.103:0/839092482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 msgr2=0x55b7a6528210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 --2- 172.21.15.103:0/839092482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 0x55b7a6528210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55b7a6535bc0 tx=0x7f42ac005f90 comp rx=0 tx=0).stop 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 -- 172.21.15.103:0/839092482 shutdown_connections 2024-09-16T02:05:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 --2- 172.21.15.103:0/839092482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 --2- 172.21.15.103:0/839092482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 0x55b7a6528210 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 --2- 172.21.15.103:0/839092482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 0x55b7a65488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 -- 172.21.15.103:0/839092482 >> 172.21.15.103:0/839092482 conn(0x55b7a65288c0 msgr2=0x55b7a6532eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 -- 172.21.15.103:0/839092482 shutdown_connections 2024-09-16T02:05:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 -- 172.21.15.103:0/839092482 wait complete. 2024-09-16T02:05:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 -- start start 2024-09-16T02:05:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.180+0000 7f42c93c3740 1 Processor -- start 2024-09-16T02:05:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 0x55b7a6551ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 0x55b7a6531180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42c93c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42c93c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b7a6549500 con 0x55b7a6537510 2024-09-16T02:05:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42c93c3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b7a6548e30 con 0x55b7a652cc50 2024-09-16T02:05:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42c93c3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b7a6549bd0 con 0x55b7a652c330 2024-09-16T02:05:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42c93c3740 10 client.0 _ll_register_callbacks cb 0x7f42b8004e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 0x55b7a6551ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 0x55b7a6551ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:35290/0 (socket says 172.21.15.103:35290) 2024-09-16T02:05:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42bfe00640 1 -- 172.21.15.103:0/3558352279 learned_addr learned my addr 172.21.15.103:0/3558352279 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.181+0000 7f42bea00640 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 0x55b7a6531180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42bfe00640 1 -- 172.21.15.103:0/3558352279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 msgr2=0x55b7a6551ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42bfe00640 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 0x55b7a6551ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42bfe00640 1 -- 172.21.15.103:0/3558352279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 msgr2=0x55b7a6531180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42bfe00640 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 0x55b7a6531180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42bfe00640 1 -- 172.21.15.103:0/3558352279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a6531cb0 con 0x55b7a6537510 2024-09-16T02:05:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42bfe00640 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65316c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f42b8005c40 tx=0x7f42b8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ed53453437ff47 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42a9600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:05:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42a9600640 1 -- 172.21.15.103:0/3558352279 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8029040 con 0x55b7a6537510 2024-09-16T02:05:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42a9600640 1 -- 172.21.15.103:0/3558352279 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7f42b8019430 con 0x55b7a6537510 2024-09-16T02:05:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42c93c3740 1 -- 172.21.15.103:0/3558352279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ac004490 con 0x55b7a6537510 2024-09-16T02:05:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42a9600640 1 -- 172.21.15.103:0/3558352279 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f42b8019630 con 0x55b7a6537510 2024-09-16T02:05:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42c93c3740 10 client.15036 fetch_fsmap learned FSMap version 23 2024-09-16T02:05:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.182+0000 7f42c93c3740 1 -- 172.21.15.103:0/3558352279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b7a6550810 con 0x55b7a6537510 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.183+0000 7f42a9600640 1 -- 172.21.15.103:0/3558352279 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42ac004490 con 0x55b7a6537510 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.183+0000 7f42a9600640 1 -- 172.21.15.103:0/3558352279 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f42b80225b0 con 0x55b7a6537510 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.183+0000 7f42c93c3740 10 client.15036 fetch_fsmap finished waiting for FSMap version 23 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.183+0000 7f42c93c3740 10 client.15036 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.183+0000 7f42c93c3740 20 client.15036 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.183+0000 7f42c93c3740 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55b7a65b7190 0x55b7a65b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.183+0000 7f42c93c3740 4 client.15036 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.183+0000 7f42c93c3740 1 -- 172.21.15.103:0/3558352279 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "status"}) -- 0x55b7a6499b30 con 0x55b7a65b7190 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.184+0000 7f42bea00640 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55b7a65b7190 0x55b7a65b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.184+0000 7f42bea00640 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55b7a65b7190 0x55b7a65b5ea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9752f17f2fb9ff3 server_cookie=8484f9a75a32f2bf in_seq=0 out_seq=0 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.184+0000 7f42a9600640 10 client.15036 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.184+0000 7f42a9600640 1 -- 172.21.15.103:0/3558352279 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b7a6499b30 con 0x55b7a65b7190 2024-09-16T02:05:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.184+0000 7f42a9600640 10 client.15036 handle_command_reply: tid=0 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.184+0000 7f42c93c3740 1 client.15036 shutdown 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.184+0000 7f42c93c3740 1 -- 172.21.15.103:0/3558352279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 msgr2=0x55b7a65316c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.184+0000 7f42c93c3740 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65316c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f42b8005c40 tx=0x7f42b8011810 comp rx=0 tx=0).stop 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c93c3740 1 -- 172.21.15.103:0/3558352279 shutdown_connections 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c93c3740 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55b7a65b7190 0x55b7a65b5ea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c93c3740 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7a6537510 0x55b7a65316c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c93c3740 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7a652cc50 0x55b7a6531180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c93c3740 1 --2- 172.21.15.103:0/3558352279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b7a652c330 0x55b7a6551ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c93c3740 1 -- 172.21.15.103:0/3558352279 >> 172.21.15.103:0/3558352279 conn(0x55b7a65288c0 msgr2=0x55b7a65336d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42a9600640 0 client.15036 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c93c3740 1 -- 172.21.15.103:0/3558352279 shutdown_connections 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c93c3740 1 -- 172.21.15.103:0/3558352279 wait complete. 2024-09-16T02:05:25.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c93c3740 20 client.15036 trim_cache size 0 max 16384 2024-09-16T02:05:25.192 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:05:25.192 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "a20dd7a4-73ce-11ef-bceb-c7b262605968", 2024-09-16T02:05:25.192 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-16T02:05:25.192 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24545, 2024-09-16T02:05:25.192 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-16T02:05:25.192 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-16T02:05:25.192 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-16T02:05:25.192 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 107.15382990000001, 2024-09-16T02:05:25.192 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 23, 2024-09-16T02:05:25.193 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 79, 2024-09-16T02:05:25.193 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 77, 2024-09-16T02:05:25.193 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 119.943922094 2024-09-16T02:05:25.193 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:05:25.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c6800640 1 -- 172.21.15.103:0/3010876853 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f42a40a3350 msgr2=0x7f42a40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c6800640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f42a40a3350 0x7f42a40a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f42b00ed750 tx=0x7f42b002a410 comp rx=0 tx=0).stop 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c6800640 1 -- 172.21.15.103:0/3010876853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c0101de0 msgr2=0x7f42c019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.185+0000 7f42c6800640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c0101de0 0x7f42c019a670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42b0006010 tx=0x7f42b0004040 comp rx=0 tx=0).stop 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.186+0000 7f42c6800640 1 -- 172.21.15.103:0/3010876853 shutdown_connections 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.186+0000 7f42c6800640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f42a40a3350 0x7f42a40a5810 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.186+0000 7f42c6800640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c0106f10 0x7f42c019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.186+0000 7f42c6800640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c0102790 0x7f42c019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.186+0000 7f42c6800640 1 --2- 172.21.15.103:0/3010876853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c0101de0 0x7f42c019a670 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.186+0000 7f42c6800640 1 -- 172.21.15.103:0/3010876853 >> 172.21.15.103:0/3010876853 conn(0x7f42c00fd8c0 msgr2=0x7f42c00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.186+0000 7f42c6800640 1 -- 172.21.15.103:0/3010876853 shutdown_connections 2024-09-16T02:05:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:25.186+0000 7f42c6800640 1 -- 172.21.15.103:0/3010876853 wait complete. 2024-09-16T02:05:25.986 INFO:teuthology.run_tasks:Running task sequential... 2024-09-16T02:05:25.996 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-16T02:05:25.996 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-16T02:05:25.997 INFO:teuthology.task.print:Enabling mgr modules 2024-09-16T02:05:25.997 INFO:teuthology.run_tasks:Running task kclient... 2024-09-16T02:05:26.043 INFO:tasks.kclient:Mounting kernel clients... 2024-09-16T02:05:26.043 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-16T02:05:26.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:05:26.044 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-16T02:05:26.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:05:26.081 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-09-16T02:05:26.181 INFO:teuthology.orchestra.run.smithi103.stderr:Cannot find device "ceph-brx" 2024-09-16T02:05:26.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:05:26.184 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-16T02:05:26.184 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T02:05:26.184 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T02:05:26.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T02:05:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.049+0000 7ff021200640 1 -- 172.21.15.103:0/1753889626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01c112870 msgr2=0x7ff01c114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:27.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.049+0000 7ff021200640 1 --2- 172.21.15.103:0/1753889626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01c112870 0x7ff01c114c60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff014009ca0 tx=0x7ff01402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.049+0000 7ff021200640 1 -- 172.21.15.103:0/1753889626 shutdown_connections 2024-09-16T02:05:27.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.049+0000 7ff021200640 1 --2- 172.21.15.103:0/1753889626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01c112870 0x7ff01c114c60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.049+0000 7ff021200640 1 --2- 172.21.15.103:0/1753889626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff01c077040 0x7ff01c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.049+0000 7ff021200640 1 --2- 172.21.15.103:0/1753889626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01c076720 0x7ff01c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.049+0000 7ff021200640 1 -- 172.21.15.103:0/1753889626 >> 172.21.15.103:0/1753889626 conn(0x7ff01c0ff8d0 msgr2=0x7ff01c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:27.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.049+0000 7ff021200640 1 -- 172.21.15.103:0/1753889626 shutdown_connections 2024-09-16T02:05:27.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.049+0000 7ff021200640 1 -- 172.21.15.103:0/1753889626 wait complete. 2024-09-16T02:05:27.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.050+0000 7ff021200640 1 Processor -- start 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.050+0000 7ff021200640 1 -- start start 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff021200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01c076720 0x7ff01c1a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff021200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff01c077040 0x7ff01c1a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff021200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01c112870 0x7ff01c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff021200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c118400 con 0x7ff01c112870 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff021200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c118280 con 0x7ff01c077040 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff021200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c118580 con 0x7ff01c076720 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff01be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01c112870 0x7ff01c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff01be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01c112870 0x7ff01c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45206/0 (socket says 172.21.15.103:45206) 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff01be00640 1 -- 172.21.15.103:0/317416184 learned_addr learned my addr 172.21.15.103:0/317416184 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:27.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff013e00640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff01c077040 0x7ff01c1a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:27.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff01b400640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01c076720 0x7ff01c1a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:27.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff01be00640 1 -- 172.21.15.103:0/317416184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01c076720 msgr2=0x7ff01c1a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:27.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff01be00640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01c076720 0x7ff01c1a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff01be00640 1 -- 172.21.15.103:0/317416184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff01c077040 msgr2=0x7ff01c1a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:27.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff01be00640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff01c077040 0x7ff01c1a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.051+0000 7ff01be00640 1 -- 172.21.15.103:0/317416184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c19ef20 con 0x7ff01c112870 2024-09-16T02:05:27.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.052+0000 7ff01be00640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01c112870 0x7ff01c19e6b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff014005e30 tx=0x7ff014003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:27.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.052+0000 7ff018c00640 1 -- 172.21.15.103:0/317416184 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014031650 con 0x7ff01c112870 2024-09-16T02:05:27.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.052+0000 7ff018c00640 1 -- 172.21.15.103:0/317416184 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0140317f0 con 0x7ff01c112870 2024-09-16T02:05:27.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.052+0000 7ff021200640 1 -- 172.21.15.103:0/317416184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c19f1b0 con 0x7ff01c112870 2024-09-16T02:05:27.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.052+0000 7ff018c00640 1 -- 172.21.15.103:0/317416184 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01402fa10 con 0x7ff01c112870 2024-09-16T02:05:27.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.052+0000 7ff021200640 1 -- 172.21.15.103:0/317416184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c1ab560 con 0x7ff01c112870 2024-09-16T02:05:27.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.053+0000 7ff021200640 1 -- 172.21.15.103:0/317416184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff01c19f570 con 0x7ff01c112870 2024-09-16T02:05:27.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.055+0000 7ff018c00640 1 -- 172.21.15.103:0/317416184 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff01402fbd0 con 0x7ff01c112870 2024-09-16T02:05:27.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.056+0000 7ff018c00640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0000a3350 0x7ff0000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:27.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.056+0000 7ff018c00640 1 -- 172.21.15.103:0/317416184 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8248+0+0 (secure 0 0 0) 0x7ff0140f0ca0 con 0x7ff01c112870 2024-09-16T02:05:27.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.056+0000 7ff01b400640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0000a3350 0x7ff0000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:27.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.057+0000 7ff01b400640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0000a3350 0x7ff0000a5810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff00c005d00 tx=0x7ff00c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:27.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.060+0000 7ff018c00640 1 -- 172.21.15.103:0/317416184 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0140bd900 con 0x7ff01c112870 2024-09-16T02:05:27.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.271+0000 7ff021200640 1 -- 172.21.15.103:0/317416184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff01c10da00 con 0x7ff01c112870 2024-09-16T02:05:27.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.271+0000 7ff018c00640 1 -- 172.21.15.103:0/317416184 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7ff0140319a0 con 0x7ff01c112870 2024-09-16T02:05:27.274 INFO:teuthology.orchestra.run.smithi103.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T02:05:27.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.273+0000 7ff021200640 1 -- 172.21.15.103:0/317416184 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0000a3350 msgr2=0x7ff0000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:27.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.273+0000 7ff021200640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0000a3350 0x7ff0000a5810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff00c005d00 tx=0x7ff00c00b410 comp rx=0 tx=0).stop 2024-09-16T02:05:27.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.274+0000 7ff021200640 1 -- 172.21.15.103:0/317416184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01c112870 msgr2=0x7ff01c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:27.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.274+0000 7ff021200640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01c112870 0x7ff01c19e6b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff014005e30 tx=0x7ff014003f10 comp rx=0 tx=0).stop 2024-09-16T02:05:27.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.274+0000 7ff021200640 1 -- 172.21.15.103:0/317416184 shutdown_connections 2024-09-16T02:05:27.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.274+0000 7ff021200640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff0000a3350 0x7ff0000a5810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.274+0000 7ff021200640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01c112870 0x7ff01c19e6b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.274+0000 7ff021200640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff01c077040 0x7ff01c1a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.274+0000 7ff021200640 1 --2- 172.21.15.103:0/317416184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01c076720 0x7ff01c1a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:27.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.274+0000 7ff021200640 1 -- 172.21.15.103:0/317416184 >> 172.21.15.103:0/317416184 conn(0x7ff01c0ff8d0 msgr2=0x7ff01c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:27.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.274+0000 7ff021200640 1 -- 172.21.15.103:0/317416184 shutdown_connections 2024-09-16T02:05:27.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:27.274+0000 7ff021200640 1 -- 172.21.15.103:0/317416184 wait complete. 2024-09-16T02:05:27.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:27 smithi172 ceph-mon[35340]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:27.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:27 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/317416184' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T02:05:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:27 smithi103 ceph-mon[30398]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:27.976 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:27 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/317416184' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T02:05:28.013 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T02:05:28.013 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T02:05:28.013 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2024-09-16T02:05:28.013 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T02:05:28.013 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T02:05:28.013 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T02:05:28.013 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T02:05:28.014 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T02:05:28.014 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T02:05:28.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:05:28.014 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2024-09-16T02:05:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:27 smithi136 ceph-mon[35323]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:05:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:27 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/317416184' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T02:05:28.050 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T02:05:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T02:05:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T02:05:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-16T02:05:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f0 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f1 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T02:05:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42125sec preferred_lft 42125sec 2024-09-16T02:05:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-16T02:05:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-16T02:05:28.052 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T02:05:28.052 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T02:05:28.052 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-16T02:05:28.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add name ceph-brx type bridge 2024-09-16T02:05:28.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr flush dev ceph-brx 2024-09-16T02:05:28.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set ceph-brx up 2024-09-16T02:05:28.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T02:05:28.053 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-16T02:05:28.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:05:28.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:05:28.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:05:28.528 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T02:05:28.583 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-09-16T02:05:28.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:05:28.587 DEBUG:teuthology.orchestra.run.smithi103:> route 2024-09-16T02:05:28.637 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2024-09-16T02:05:28.637 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T02:05:28.638 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T02:05:28.638 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T02:05:28.638 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T02:05:28.638 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T02:05:28.638 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-16T02:05:28.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T02:05:28.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T02:05:28.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T02:05:28.639 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-16T02:05:28.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:05:29.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:05:29.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:05:29.207 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-16T02:05:29.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:05:29.243 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2024-09-16T02:05:29.319 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T02:05:29.319 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-16T02:05:29.320 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T02:05:29.320 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T02:05:29.320 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-16T02:05:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:05:29.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:05:29.497 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T02:05:29.497 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T02:05:29.497 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-16T02:05:29.497 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T02:05:29.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T02:05:29.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T02:05:29.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T02:05:29.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T02:05:29.498 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-16T02:05:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:05:29.708 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:29 smithi103 ceph-mon[30398]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:29.708 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-09-16T02:05:29.708 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-09-16T02:05:29.709 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-16T02:05:29.709 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:29 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-09-16T02:05:29.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:05:29.744 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T02:05:29.744 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-16T02:05:29.744 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.0 up 2024-09-16T02:05:29.744 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T02:05:29.744 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-16T02:05:29.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:29 smithi172 ceph-mon[35340]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:29.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-09-16T02:05:29.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-09-16T02:05:29.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-16T02:05:29.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:29 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-09-16T02:05:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:05:29.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:05:29.913 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T02:05:29.913 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T02:05:29.949 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T02:05:29.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T02:05:29.949 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T02:05:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:29 smithi136 ceph-mon[35323]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T02:05:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-09-16T02:05:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-09-16T02:05:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-16T02:05:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:29 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-09-16T02:05:30.025 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-16T02:05:30.025 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-16T02:05:30.025 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/1d178de4-9eb9-4acc-93f7-1d9d1e58235e 2024-09-16T02:05:30.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:05:30.026 DEBUG:teuthology.orchestra.run.smithi103:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/1d178de4-9eb9-4acc-93f7-1d9d1e58235e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-16T02:05:30.175 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-16T02:05:30.175 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-16T02:05:30.742 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-16T02:05:30.742 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-16T02:05:30.742 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: resolved to: "172.21.15.103:6789,172.21.15.136:6789,172.21.15.172:6789" 2024-09-16T02:05:30.742 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: trying mount with new device syntax: 0@a20dd7a4-73ce-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/1d178de4-9eb9-4acc-93f7-1d9d1e58235e 2024-09-16T02:05:30.742 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.103:6789/172.21.15.136:6789/172.21.15.172:6789" will pass to kernel 2024-09-16T02:05:30.745 INFO:teuthology.orchestra.run.smithi103.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T02:05:30.745 INFO:teuthology.orchestra.run.smithi103.stdout: You just mounted a file system that supports labels which does not 2024-09-16T02:05:30.745 INFO:teuthology.orchestra.run.smithi103.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T02:05:30.746 INFO:teuthology.orchestra.run.smithi103.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T02:05:30.746 INFO:teuthology.orchestra.run.smithi103.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T02:05:30.748 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T02:05:30.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:05:30.748 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T02:05:30.797 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-09-16T02:05:30.797 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-16T02:05:30.797 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-16T02:05:30.797 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:30 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-09-16T02:05:30.797 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:30 smithi103 ceph-mon[30398]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T02:05:30.797 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:30 smithi103 ceph-mon[30398]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T02:05:30.807 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T02:05:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T02:05:30.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:05:30.901 INFO:teuthology.orchestra.run.smithi103.stdout:client24982 2024-09-16T02:05:30.901 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T02:05:30.901 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T02:05:30.938 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T02:05:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-09-16T02:05:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-16T02:05:31.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-16T02:05:31.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:30 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-09-16T02:05:31.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:30 smithi136 ceph-mon[35323]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T02:05:31.024 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:30 smithi136 ceph-mon[35323]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T02:05:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-09-16T02:05:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-16T02:05:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-16T02:05:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:30 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-09-16T02:05:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:30 smithi172 ceph-mon[35340]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T02:05:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:30 smithi172 ceph-mon[35340]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T02:05:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T02:05:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:05:31.069 INFO:teuthology.orchestra.run.smithi103.stdout:client24982 2024-09-16T02:05:31.070 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T02:05:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T02:05:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:05:31.160 INFO:teuthology.orchestra.run.smithi103.stdout:client24982 2024-09-16T02:05:31.161 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968.client24982/status') 2024-09-16T02:05:31.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T02:05:31.262 INFO:teuthology.orchestra.run.smithi103.stdout:instance: client.24982 (3)172.21.15.103:0/1786703416 2024-09-16T02:05:31.262 INFO:teuthology.orchestra.run.smithi103.stdout:blocklisted: false 2024-09-16T02:05:31.262 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records in 2024-09-16T02:05:31.262 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records out 2024-09-16T02:05:31.262 INFO:teuthology.orchestra.run.smithi103.stderr:72 bytes copied, 5.3607e-05 s, 1.3 MB/s 2024-09-16T02:05:31.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T02:05:31.272 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T02:05:31.282 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T02:05:31.283 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-16T02:05:31.283 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-16T02:05:31.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.436+0000 7f586c800640 1 -- 172.21.15.103:0/1881606374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5864103d60 msgr2=0x7f5864104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:31.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.436+0000 7f586c800640 1 --2- 172.21.15.103:0/1881606374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5864103d60 0x7f5864104160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5858009900 tx=0x7f585802a410 comp rx=0 tx=0).stop 2024-09-16T02:05:31.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.436+0000 7f586c800640 1 -- 172.21.15.103:0/1881606374 shutdown_connections 2024-09-16T02:05:31.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.436+0000 7f586c800640 1 --2- 172.21.15.103:0/1881606374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5864105910 0x7f586410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:31.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.436+0000 7f586c800640 1 --2- 172.21.15.103:0/1881606374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5864104f50 0x7f58641053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:31.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.436+0000 7f586c800640 1 --2- 172.21.15.103:0/1881606374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5864103d60 0x7f5864104160 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:31.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.436+0000 7f586c800640 1 -- 172.21.15.103:0/1881606374 >> 172.21.15.103:0/1881606374 conn(0x7f58640ff4e0 msgr2=0x7f5864101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:31.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.436+0000 7f586c800640 1 -- 172.21.15.103:0/1881606374 shutdown_connections 2024-09-16T02:05:31.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.436+0000 7f586c800640 1 -- 172.21.15.103:0/1881606374 wait complete. 2024-09-16T02:05:31.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.437+0000 7f586c800640 1 Processor -- start 2024-09-16T02:05:31.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.437+0000 7f586c800640 1 -- start start 2024-09-16T02:05:31.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586c800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5864103d60 0x7f5864199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:31.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586c800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5864104f50 0x7f586419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:31.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586c800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5864105910 0x7f58641a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:31.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586c800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f586410fa10 con 0x7f5864105910 2024-09-16T02:05:31.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586c800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f586410f890 con 0x7f5864104f50 2024-09-16T02:05:31.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586c800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f586410fb90 con 0x7f5864103d60 2024-09-16T02:05:31.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5864105910 0x7f58641a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:31.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5864105910 0x7f58641a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54426/0 (socket says 172.21.15.103:54426) 2024-09-16T02:05:31.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586a800640 1 -- 172.21.15.103:0/1963755628 learned_addr learned my addr 172.21.15.103:0/1963755628 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:31.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f5869400640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5864104f50 0x7f586419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:31.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f5869e00640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5864103d60 0x7f5864199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:31.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586a800640 1 -- 172.21.15.103:0/1963755628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5864103d60 msgr2=0x7f5864199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:31.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586a800640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5864103d60 0x7f5864199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:31.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586a800640 1 -- 172.21.15.103:0/1963755628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5864104f50 msgr2=0x7f586419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:31.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586a800640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5864104f50 0x7f586419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:31.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.438+0000 7f586a800640 1 -- 172.21.15.103:0/1963755628 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58641a1bf0 con 0x7f5864105910 2024-09-16T02:05:31.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.439+0000 7f586a800640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5864105910 0x7f58641a1580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5860009fd0 tx=0x7f586000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:31.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.439+0000 7f585ea00640 1 -- 172.21.15.103:0/1963755628 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860017070 con 0x7f5864105910 2024-09-16T02:05:31.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.439+0000 7f585ea00640 1 -- 172.21.15.103:0/1963755628 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5860003e50 con 0x7f5864105910 2024-09-16T02:05:31.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.439+0000 7f586c800640 1 -- 172.21.15.103:0/1963755628 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58641a1ee0 con 0x7f5864105910 2024-09-16T02:05:31.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.439+0000 7f585ea00640 1 -- 172.21.15.103:0/1963755628 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58600106c0 con 0x7f5864105910 2024-09-16T02:05:31.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.439+0000 7f586c800640 1 -- 172.21.15.103:0/1963755628 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58641a23a0 con 0x7f5864105910 2024-09-16T02:05:31.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.440+0000 7f586c800640 1 -- 172.21.15.103:0/1963755628 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586410b080 con 0x7f5864105910 2024-09-16T02:05:31.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.442+0000 7f585ea00640 1 -- 172.21.15.103:0/1963755628 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f5860003970 con 0x7f5864105910 2024-09-16T02:05:31.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.443+0000 7f585ea00640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f584c0a3370 0x7f584c0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:31.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.443+0000 7f585ea00640 1 -- 172.21.15.103:0/1963755628 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f58600cf5e0 con 0x7f5864105910 2024-09-16T02:05:31.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.443+0000 7f5869e00640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f584c0a3370 0x7f584c0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:31.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.444+0000 7f5869e00640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f584c0a3370 0x7f584c0a5830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f585802aa00 tx=0x7f58580315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:31.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.447+0000 7f585ea00640 1 -- 172.21.15.103:0/1963755628 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f586009bbd0 con 0x7f5864105910 2024-09-16T02:05:31.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:31.659+0000 7f586c800640 1 -- 172.21.15.103:0/1963755628 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f5864062e90 con 0x7f5864105910 2024-09-16T02:05:32.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:31 smithi103 ceph-mon[30398]: pgmap v341: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-16T02:05:32.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:31 smithi103 ceph-mon[30398]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T02:05:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:31 smithi136 ceph-mon[35323]: pgmap v341: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-16T02:05:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:31 smithi136 ceph-mon[35323]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T02:05:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:31 smithi172 ceph-mon[35340]: pgmap v341: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-16T02:05:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:31 smithi172 ceph-mon[35340]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T02:05:32.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.555+0000 7f585ea00640 1 -- 172.21.15.103:0/1963755628 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f586009fed0 con 0x7f5864105910 2024-09-16T02:05:32.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 -- 172.21.15.103:0/1963755628 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f584c0a3370 msgr2=0x7f584c0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f584c0a3370 0x7f584c0a5830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f585802aa00 tx=0x7f58580315b0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 -- 172.21.15.103:0/1963755628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5864105910 msgr2=0x7f58641a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5864105910 0x7f58641a1580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5860009fd0 tx=0x7f586000b260 comp rx=0 tx=0).stop 2024-09-16T02:05:32.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 -- 172.21.15.103:0/1963755628 shutdown_connections 2024-09-16T02:05:32.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f584c0a3370 0x7f584c0a5830 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5864105910 0x7f58641a1580 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5864104f50 0x7f586419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 --2- 172.21.15.103:0/1963755628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5864103d60 0x7f5864199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 -- 172.21.15.103:0/1963755628 >> 172.21.15.103:0/1963755628 conn(0x7f58640ff4e0 msgr2=0x7f5864108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:32.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 -- 172.21.15.103:0/1963755628 shutdown_connections 2024-09-16T02:05:32.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.558+0000 7f586c800640 1 -- 172.21.15.103:0/1963755628 wait complete. 2024-09-16T02:05:32.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-16T02:05:32.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.732+0000 7f49c8000640 1 -- 172.21.15.103:0/354267127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49c01099d0 msgr2=0x7f49c010be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.732+0000 7f49c8000640 1 --2- 172.21.15.103:0/354267127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49c01099d0 0x7f49c010be90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009ca0 tx=0x7f49b802a950 comp rx=0 tx=0).stop 2024-09-16T02:05:32.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.732+0000 7f49c8000640 1 -- 172.21.15.103:0/354267127 shutdown_connections 2024-09-16T02:05:32.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.732+0000 7f49c8000640 1 --2- 172.21.15.103:0/354267127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49c01099d0 0x7f49c010be90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.732+0000 7f49c8000640 1 --2- 172.21.15.103:0/354267127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49c0074bf0 0x7f49c0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.732+0000 7f49c8000640 1 --2- 172.21.15.103:0/354267127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c00761d0 0x7f49c0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.732+0000 7f49c8000640 1 -- 172.21.15.103:0/354267127 >> 172.21.15.103:0/354267127 conn(0x7f49c00fd2e0 msgr2=0x7f49c00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:32.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.733+0000 7f49c8000640 1 -- 172.21.15.103:0/354267127 shutdown_connections 2024-09-16T02:05:32.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.733+0000 7f49c8000640 1 -- 172.21.15.103:0/354267127 wait complete. 2024-09-16T02:05:32.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.733+0000 7f49c8000640 1 Processor -- start 2024-09-16T02:05:32.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c8000640 1 -- start start 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c8000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c0074bf0 0x7f49c0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c8000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49c00761d0 0x7f49c019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c8000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49c01099d0 0x7f49c01a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c8000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f49c010fa20 con 0x7f49c00761d0 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c8000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49c010f8a0 con 0x7f49c01099d0 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c8000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49c010fba0 con 0x7f49c0074bf0 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c4c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49c00761d0 0x7f49c019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c0074bf0 0x7f49c0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c0074bf0 0x7f49c0199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:48688/0 (socket says 172.21.15.103:48688) 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c5600640 1 -- 172.21.15.103:0/3998436631 learned_addr learned my addr 172.21.15.103:0/3998436631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.734+0000 7f49c6000640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49c01099d0 0x7f49c01a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49c5600640 1 -- 172.21.15.103:0/3998436631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49c01099d0 msgr2=0x7f49c01a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49c5600640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49c01099d0 0x7f49c01a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49c5600640 1 -- 172.21.15.103:0/3998436631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49c00761d0 msgr2=0x7f49c019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49c5600640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49c00761d0 0x7f49c019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49c5600640 1 -- 172.21.15.103:0/3998436631 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c01a1bd0 con 0x7f49c0074bf0 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49c5600640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c0074bf0 0x7f49c0199fb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f49b400e9f0 tx=0x7f49b400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49be000640 1 -- 172.21.15.103:0/3998436631 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b400cd80 con 0x7f49c0074bf0 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49be000640 1 -- 172.21.15.103:0/3998436631 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b4005500 con 0x7f49c0074bf0 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49be000640 1 -- 172.21.15.103:0/3998436631 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4010690 con 0x7f49c0074bf0 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49c8000640 1 -- 172.21.15.103:0/3998436631 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c01a1ec0 con 0x7f49c0074bf0 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.735+0000 7f49c8000640 1 -- 172.21.15.103:0/3998436631 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c01a2320 con 0x7f49c0074bf0 2024-09-16T02:05:32.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.736+0000 7f49c8000640 1 -- 172.21.15.103:0/3998436631 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c0108790 con 0x7f49c0074bf0 2024-09-16T02:05:32.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.738+0000 7f49be000640 1 -- 172.21.15.103:0/3998436631 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f49b40046c0 con 0x7f49c0074bf0 2024-09-16T02:05:32.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.739+0000 7f49be000640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49a80a3370 0x7f49a80a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:32.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.740+0000 7f49be000640 1 -- 172.21.15.103:0/3998436631 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f49b40cfff0 con 0x7f49c0074bf0 2024-09-16T02:05:32.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.740+0000 7f49c4c00640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49a80a3370 0x7f49a80a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:32.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.740+0000 7f49c4c00640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49a80a3370 0x7f49a80a5830 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f49c019b4c0 tx=0x7f49ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:32.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.742+0000 7f49be000640 1 -- 172.21.15.103:0/3998436631 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49b409cc70 con 0x7f49c0074bf0 2024-09-16T02:05:32.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.942+0000 7f49c8000640 1 -- 172.21.15.103:0/3998436631 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f49c0062630 con 0x7f49a80a3370 2024-09-16T02:05:32.951 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:32 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1963755628' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-16T02:05:32.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.963+0000 7f49be000640 1 -- 172.21.15.103:0/3998436631 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f49c0062630 con 0x7f49a80a3370 2024-09-16T02:05:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:{} 2024-09-16T02:05:32.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 -- 172.21.15.103:0/3998436631 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49a80a3370 msgr2=0x7f49a80a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49a80a3370 0x7f49a80a5830 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f49c019b4c0 tx=0x7f49ac005f90 comp rx=0 tx=0).stop 2024-09-16T02:05:32.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 -- 172.21.15.103:0/3998436631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c0074bf0 msgr2=0x7f49c0199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:32.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c0074bf0 0x7f49c0199fb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f49b400e9f0 tx=0x7f49b400edf0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 -- 172.21.15.103:0/3998436631 shutdown_connections 2024-09-16T02:05:32.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f49a80a3370 0x7f49a80a5830 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49c01099d0 0x7f49c01a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49c00761d0 0x7f49c019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 --2- 172.21.15.103:0/3998436631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c0074bf0 0x7f49c0199fb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:32.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 -- 172.21.15.103:0/3998436631 >> 172.21.15.103:0/3998436631 conn(0x7f49c00fd2e0 msgr2=0x7f49c0105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:32.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 -- 172.21.15.103:0/3998436631 shutdown_connections 2024-09-16T02:05:32.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:32.965+0000 7f49c8000640 1 -- 172.21.15.103:0/3998436631 wait complete. 2024-09-16T02:05:32.983 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-16T02:05:33.000 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-16T02:05:33.042 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T02:05:33.043 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:32 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1963755628' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-16T02:05:33.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:32 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1963755628' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-16T02:05:33.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.196+0000 7f7f76e00640 1 -- 172.21.15.103:0/172702926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f70103d30 msgr2=0x7f7f70104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.196+0000 7f7f76e00640 1 --2- 172.21.15.103:0/172702926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f70103d30 0x7f7f70104130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009900 tx=0x7f7f6002a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.196+0000 7f7f76e00640 1 -- 172.21.15.103:0/172702926 shutdown_connections 2024-09-16T02:05:33.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.196+0000 7f7f76e00640 1 --2- 172.21.15.103:0/172702926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f701058e0 0x7f7f7010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.196+0000 7f7f76e00640 1 --2- 172.21.15.103:0/172702926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f70104f20 0x7f7f701053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.196+0000 7f7f76e00640 1 --2- 172.21.15.103:0/172702926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f70103d30 0x7f7f70104130 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.196+0000 7f7f76e00640 1 -- 172.21.15.103:0/172702926 >> 172.21.15.103:0/172702926 conn(0x7f7f700ff4b0 msgr2=0x7f7f701018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:33.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.196+0000 7f7f76e00640 1 -- 172.21.15.103:0/172702926 shutdown_connections 2024-09-16T02:05:33.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.196+0000 7f7f76e00640 1 -- 172.21.15.103:0/172702926 wait complete. 2024-09-16T02:05:33.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.197+0000 7f7f76e00640 1 Processor -- start 2024-09-16T02:05:33.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.197+0000 7f7f76e00640 1 -- start start 2024-09-16T02:05:33.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.197+0000 7f7f76e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f70103d30 0x7f7f7019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f76e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f70104f20 0x7f7f7019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f76e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f701058e0 0x7f7f701a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f76e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7010f9b0 con 0x7f7f70103d30 2024-09-16T02:05:33.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f76e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7010f830 con 0x7f7f701058e0 2024-09-16T02:05:33.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f76e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7010fb30 con 0x7f7f70104f20 2024-09-16T02:05:33.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f6fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f70103d30 0x7f7f7019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f6fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f70103d30 0x7f7f7019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54482/0 (socket says 172.21.15.103:54482) 2024-09-16T02:05:33.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f6fe00640 1 -- 172.21.15.103:0/1144704001 learned_addr learned my addr 172.21.15.103:0/1144704001 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:33.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f74e00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f701058e0 0x7f7f701a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f6f400640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f70104f20 0x7f7f7019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f6fe00640 1 -- 172.21.15.103:0/1144704001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f70104f20 msgr2=0x7f7f7019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f6fe00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f70104f20 0x7f7f7019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f6fe00640 1 -- 172.21.15.103:0/1144704001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f701058e0 msgr2=0x7f7f701a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f6fe00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f701058e0 0x7f7f701a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f6fe00640 1 -- 172.21.15.103:0/1144704001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f701a1c50 con 0x7f7f70103d30 2024-09-16T02:05:33.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.198+0000 7f7f74e00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f701058e0 0x7f7f701a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:33.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.199+0000 7f7f6fe00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f70103d30 0x7f7f7019a030 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7f60031610 tx=0x7f7f60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:33.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.199+0000 7f7f6cc00640 1 -- 172.21.15.103:0/1144704001 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6003f030 con 0x7f7f70103d30 2024-09-16T02:05:33.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.199+0000 7f7f6cc00640 1 -- 172.21.15.103:0/1144704001 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f60003ad0 con 0x7f7f70103d30 2024-09-16T02:05:33.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.199+0000 7f7f6cc00640 1 -- 172.21.15.103:0/1144704001 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60035070 con 0x7f7f70103d30 2024-09-16T02:05:33.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.199+0000 7f7f76e00640 1 -- 172.21.15.103:0/1144704001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f701a1ee0 con 0x7f7f70103d30 2024-09-16T02:05:33.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.199+0000 7f7f76e00640 1 -- 172.21.15.103:0/1144704001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f701a2290 con 0x7f7f70103d30 2024-09-16T02:05:33.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.200+0000 7f7f76e00640 1 -- 172.21.15.103:0/1144704001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7010b0f0 con 0x7f7f70103d30 2024-09-16T02:05:33.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.202+0000 7f7f6cc00640 1 -- 172.21.15.103:0/1144704001 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7f60042030 con 0x7f7f70103d30 2024-09-16T02:05:33.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.203+0000 7f7f6cc00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7f540a3250 0x7f7f540a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.203+0000 7f7f6cc00640 1 -- 172.21.15.103:0/1144704001 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f7f600f0080 con 0x7f7f70103d30 2024-09-16T02:05:33.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.203+0000 7f7f6f400640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7f540a3250 0x7f7f540a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.204+0000 7f7f6f400640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7f540a3250 0x7f7f540a5710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7f7019b540 tx=0x7f7f5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:33.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.207+0000 7f7f6cc00640 1 -- 172.21.15.103:0/1144704001 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f600bcd20 con 0x7f7f70103d30 2024-09-16T02:05:33.254 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:05:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:05:32.948+0000 7fce31200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:05:33.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:05:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:05:32.948+0000 7fce31200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:05:33.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:05:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:05:32.948+0000 7fce31200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:05:33.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:05:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:05:32.948+0000 7fce31200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:05:33.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:05:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:05:32.948+0000 7fce31200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T02:05:33.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.383+0000 7f7f76e00640 1 -- 172.21.15.103:0/1144704001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f70062f40 con 0x7f7f70103d30 2024-09-16T02:05:33.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.385+0000 7f7f6cc00640 1 -- 172.21.15.103:0/1144704001 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23048 (secure 0 0 0) 0x7f7f600bc620 con 0x7f7f70103d30 2024-09-16T02:05:33.387 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:33.387 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":81,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:05:30.540044+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":10,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":7,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:05:33.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 -- 172.21.15.103:0/1144704001 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7f540a3250 msgr2=0x7f7f540a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7f540a3250 0x7f7f540a5710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7f7019b540 tx=0x7f7f5c005f90 comp rx=0 tx=0).stop 2024-09-16T02:05:33.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 -- 172.21.15.103:0/1144704001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f70103d30 msgr2=0x7f7f7019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f70103d30 0x7f7f7019a030 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7f60031610 tx=0x7f7f60004270 comp rx=0 tx=0).stop 2024-09-16T02:05:33.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 -- 172.21.15.103:0/1144704001 shutdown_connections 2024-09-16T02:05:33.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7f540a3250 0x7f7f540a5710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f701058e0 0x7f7f701a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f70104f20 0x7f7f7019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 --2- 172.21.15.103:0/1144704001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f70103d30 0x7f7f7019a030 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 -- 172.21.15.103:0/1144704001 >> 172.21.15.103:0/1144704001 conn(0x7f7f700ff4b0 msgr2=0x7f7f701082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:33.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 -- 172.21.15.103:0/1144704001 shutdown_connections 2024-09-16T02:05:33.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.387+0000 7f7f76e00640 1 -- 172.21.15.103:0/1144704001 wait complete. 2024-09-16T02:05:33.406 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T01:58:50.953500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T02:03:33.662251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T02:03:36.012448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T02:05:33.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T02:05:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.564+0000 7f904b800640 1 -- 172.21.15.103:0/1241687619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9044100d20 msgr2=0x7f9044101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.564+0000 7f904b800640 1 --2- 172.21.15.103:0/1241687619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9044100d20 0x7f9044101180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9030009860 tx=0x7f903002a340 comp rx=0 tx=0).stop 2024-09-16T02:05:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.564+0000 7f904b800640 1 -- 172.21.15.103:0/1241687619 shutdown_connections 2024-09-16T02:05:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.564+0000 7f904b800640 1 --2- 172.21.15.103:0/1241687619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90441016c0 0x7f9044101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.564+0000 7f904b800640 1 --2- 172.21.15.103:0/1241687619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9044100d20 0x7f9044101180 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.564+0000 7f904b800640 1 --2- 172.21.15.103:0/1241687619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90441003e0 0x7f90441007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.564+0000 7f904b800640 1 -- 172.21.15.103:0/1241687619 >> 172.21.15.103:0/1241687619 conn(0x7f90440fc1b0 msgr2=0x7f90440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:33.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.564+0000 7f904b800640 1 -- 172.21.15.103:0/1241687619 shutdown_connections 2024-09-16T02:05:33.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.564+0000 7f904b800640 1 -- 172.21.15.103:0/1241687619 wait complete. 2024-09-16T02:05:33.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f904b800640 1 Processor -- start 2024-09-16T02:05:33.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f904b800640 1 -- start start 2024-09-16T02:05:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f904b800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90441003e0 0x7f904419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f904b800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9044100d20 0x7f904419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f904b800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90441016c0 0x7f90441a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f904b800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f904410f920 con 0x7f90441003e0 2024-09-16T02:05:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f904b800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f904410f7a0 con 0x7f90441016c0 2024-09-16T02:05:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f904b800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f904410faa0 con 0x7f9044100d20 2024-09-16T02:05:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f9048e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90441003e0 0x7f904419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f9048e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90441003e0 0x7f904419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54512/0 (socket says 172.21.15.103:54512) 2024-09-16T02:05:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.565+0000 7f9048e00640 1 -- 172.21.15.103:0/1855480516 learned_addr learned my addr 172.21.15.103:0/1855480516 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9043e00640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9044100d20 0x7f904419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9049800640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90441016c0 0x7f90441a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9048e00640 1 -- 172.21.15.103:0/1855480516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9044100d20 msgr2=0x7f904419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9048e00640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9044100d20 0x7f904419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9048e00640 1 -- 172.21.15.103:0/1855480516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90441016c0 msgr2=0x7f90441a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9048e00640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90441016c0 0x7f90441a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9048e00640 1 -- 172.21.15.103:0/1855480516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90441a1c20 con 0x7f90441003e0 2024-09-16T02:05:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9048e00640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90441003e0 0x7f904419a000 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f903800c420 tx=0x7f9038009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9041600640 1 -- 172.21.15.103:0/1855480516 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038003970 con 0x7f90441003e0 2024-09-16T02:05:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9041600640 1 -- 172.21.15.103:0/1855480516 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9038004020 con 0x7f90441003e0 2024-09-16T02:05:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f904b800640 1 -- 172.21.15.103:0/1855480516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90441a1f10 con 0x7f90441003e0 2024-09-16T02:05:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.566+0000 7f9041600640 1 -- 172.21.15.103:0/1855480516 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903800bad0 con 0x7f90441003e0 2024-09-16T02:05:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.567+0000 7f904b800640 1 -- 172.21.15.103:0/1855480516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90441a2370 con 0x7f90441003e0 2024-09-16T02:05:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.568+0000 7f904b800640 1 -- 172.21.15.103:0/1855480516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904410b2d0 con 0x7f90441003e0 2024-09-16T02:05:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.569+0000 7f9041600640 1 -- 172.21.15.103:0/1855480516 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f903801d020 con 0x7f90441003e0 2024-09-16T02:05:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.571+0000 7f9041600640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f902c0a3250 0x7f902c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.571+0000 7f9041600640 1 -- 172.21.15.103:0/1855480516 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f90380cf840 con 0x7f90441003e0 2024-09-16T02:05:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.571+0000 7f9043e00640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f902c0a3250 0x7f902c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.572+0000 7f9043e00640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f902c0a3250 0x7f902c0a5710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f903002a930 tx=0x7f9030030570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.574+0000 7f9041600640 1 -- 172.21.15.103:0/1855480516 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f903809c480 con 0x7f90441003e0 2024-09-16T02:05:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.750+0000 7f904b800640 1 -- 172.21.15.103:0/1855480516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9044062ec0 con 0x7f90441003e0 2024-09-16T02:05:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.750+0000 7f9041600640 1 -- 172.21.15.103:0/1855480516 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f903809bd80 con 0x7f90441003e0 2024-09-16T02:05:33.752 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-16T02:05:33.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.752+0000 7f904b800640 1 -- 172.21.15.103:0/1855480516 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f902c0a3250 msgr2=0x7f902c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.752+0000 7f904b800640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f902c0a3250 0x7f902c0a5710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f903002a930 tx=0x7f9030030570 comp rx=0 tx=0).stop 2024-09-16T02:05:33.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.752+0000 7f904b800640 1 -- 172.21.15.103:0/1855480516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90441003e0 msgr2=0x7f904419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:33 smithi103 ceph-mon[30398]: pgmap v343: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-09-16T02:05:33.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:33 smithi103 ceph-mon[30398]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T02:05:33.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:33 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1963755628' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-16T02:05:33.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:33 smithi103 ceph-mon[30398]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T02:05:33.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:33 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1144704001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:33.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.752+0000 7f904b800640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90441003e0 0x7f904419a000 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f903800c420 tx=0x7f9038009f90 comp rx=0 tx=0).stop 2024-09-16T02:05:33.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.753+0000 7f904b800640 1 -- 172.21.15.103:0/1855480516 shutdown_connections 2024-09-16T02:05:33.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.753+0000 7f904b800640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f902c0a3250 0x7f902c0a5710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.753+0000 7f904b800640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90441016c0 0x7f90441a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.753+0000 7f904b800640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9044100d20 0x7f904419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.753+0000 7f904b800640 1 --2- 172.21.15.103:0/1855480516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90441003e0 0x7f904419a000 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.753+0000 7f904b800640 1 -- 172.21.15.103:0/1855480516 >> 172.21.15.103:0/1855480516 conn(0x7f90440fc1b0 msgr2=0x7f90440fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:33.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.753+0000 7f904b800640 1 -- 172.21.15.103:0/1855480516 shutdown_connections 2024-09-16T02:05:33.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.753+0000 7f904b800640 1 -- 172.21.15.103:0/1855480516 wait complete. 2024-09-16T02:05:33.771 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T02:05:33.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.927+0000 7f05c7e00640 1 -- 172.21.15.103:0/4084796928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05c0077df0 msgr2=0x7f05c0078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.927+0000 7f05c7e00640 1 --2- 172.21.15.103:0/4084796928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05c0077df0 0x7f05c0078250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009900 tx=0x7f05ac02a410 comp rx=0 tx=0).stop 2024-09-16T02:05:33.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.927+0000 7f05c7e00640 1 -- 172.21.15.103:0/4084796928 shutdown_connections 2024-09-16T02:05:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.927+0000 7f05c7e00640 1 --2- 172.21.15.103:0/4084796928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c0078790 0x7f05c0079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.927+0000 7f05c7e00640 1 --2- 172.21.15.103:0/4084796928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05c0077df0 0x7f05c0078250 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.927+0000 7f05c7e00640 1 --2- 172.21.15.103:0/4084796928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05c0076bb0 0x7f05c0076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.927+0000 7f05c7e00640 1 -- 172.21.15.103:0/4084796928 >> 172.21.15.103:0/4084796928 conn(0x7f05c00ff4a0 msgr2=0x7f05c01018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.927+0000 7f05c7e00640 1 -- 172.21.15.103:0/4084796928 shutdown_connections 2024-09-16T02:05:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.927+0000 7f05c7e00640 1 -- 172.21.15.103:0/4084796928 wait complete. 2024-09-16T02:05:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c7e00640 1 Processor -- start 2024-09-16T02:05:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c7e00640 1 -- start start 2024-09-16T02:05:33.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05c0076bb0 0x7f05c019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c0077df0 0x7f05c019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c7e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05c0078790 0x7f05c01a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c7e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0113dd0 con 0x7f05c0076bb0 2024-09-16T02:05:33.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0113c50 con 0x7f05c0077df0 2024-09-16T02:05:33.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c7e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0113f50 con 0x7f05c0078790 2024-09-16T02:05:33.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05c0076bb0 0x7f05c019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05c0076bb0 0x7f05c019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54534/0 (socket says 172.21.15.103:54534) 2024-09-16T02:05:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.928+0000 7f05c5400640 1 -- 172.21.15.103:0/1274498673 learned_addr learned my addr 172.21.15.103:0/1274498673 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05c4a00640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c0077df0 0x7f05c019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05c5e00640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05c0078790 0x7f05c01a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05c5400640 1 -- 172.21.15.103:0/1274498673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05c0078790 msgr2=0x7f05c01a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05c5400640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05c0078790 0x7f05c01a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05c5400640 1 -- 172.21.15.103:0/1274498673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c0077df0 msgr2=0x7f05c019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05c5400640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c0077df0 0x7f05c019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05c5400640 1 -- 172.21.15.103:0/1274498673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c01a60d0 con 0x7f05c0076bb0 2024-09-16T02:05:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05c5400640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05c0076bb0 0x7f05c019e4b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f05b400ea60 tx=0x7f05b400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05be000640 1 -- 172.21.15.103:0/1274498673 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b400cde0 con 0x7f05c0076bb0 2024-09-16T02:05:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05be000640 1 -- 172.21.15.103:0/1274498673 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b40035f0 con 0x7f05c0076bb0 2024-09-16T02:05:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05c7e00640 1 -- 172.21.15.103:0/1274498673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c01a63c0 con 0x7f05c0076bb0 2024-09-16T02:05:33.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05be000640 1 -- 172.21.15.103:0/1274498673 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4010670 con 0x7f05c0076bb0 2024-09-16T02:05:33.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.929+0000 7f05c7e00640 1 -- 172.21.15.103:0/1274498673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c01a68e0 con 0x7f05c0076bb0 2024-09-16T02:05:33.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.931+0000 7f05c7e00640 1 -- 172.21.15.103:0/1274498673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c0110880 con 0x7f05c0076bb0 2024-09-16T02:05:33.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.933+0000 7f05be000640 1 -- 172.21.15.103:0/1274498673 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f05b40037a0 con 0x7f05c0076bb0 2024-09-16T02:05:33.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.934+0000 7f05be000640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f05a80a3250 0x7f05a80a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:33.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.934+0000 7f05be000640 1 -- 172.21.15.103:0/1274498673 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f05b40cfdd0 con 0x7f05c0076bb0 2024-09-16T02:05:33.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.934+0000 7f05c4a00640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f05a80a3250 0x7f05a80a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:33.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.935+0000 7f05c4a00640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f05a80a3250 0x7f05a80a5710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f05ac006010 tx=0x7f05ac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:33.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:33.938+0000 7f05be000640 1 -- 172.21.15.103:0/1274498673 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05b409ca70 con 0x7f05c0076bb0 2024-09-16T02:05:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:33 smithi136 ceph-mon[35323]: pgmap v343: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-09-16T02:05:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:33 smithi136 ceph-mon[35323]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T02:05:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:33 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1963755628' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-16T02:05:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:33 smithi136 ceph-mon[35323]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T02:05:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:33 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1144704001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:33 smithi172 ceph-mon[35340]: pgmap v343: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-09-16T02:05:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:33 smithi172 ceph-mon[35340]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T02:05:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:33 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1963755628' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-16T02:05:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:33 smithi172 ceph-mon[35340]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T02:05:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:33 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1144704001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:34.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.115+0000 7f05c7e00640 1 -- 172.21.15.103:0/1274498673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f05c0062df0 con 0x7f05c0076bb0 2024-09-16T02:05:34.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.115+0000 7f05be000640 1 -- 172.21.15.103:0/1274498673 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7f05b409c370 con 0x7f05c0076bb0 2024-09-16T02:05:34.118 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-09-16T02:05:34.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 -- 172.21.15.103:0/1274498673 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f05a80a3250 msgr2=0x7f05a80a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f05a80a3250 0x7f05a80a5710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f05ac006010 tx=0x7f05ac000bc0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 -- 172.21.15.103:0/1274498673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05c0076bb0 msgr2=0x7f05c019e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05c0076bb0 0x7f05c019e4b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f05b400ea60 tx=0x7f05b400ef20 comp rx=0 tx=0).stop 2024-09-16T02:05:34.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 -- 172.21.15.103:0/1274498673 shutdown_connections 2024-09-16T02:05:34.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f05a80a3250 0x7f05a80a5710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05c0078790 0x7f05c01a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c0077df0 0x7f05c019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 --2- 172.21.15.103:0/1274498673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05c0076bb0 0x7f05c019e4b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 -- 172.21.15.103:0/1274498673 >> 172.21.15.103:0/1274498673 conn(0x7f05c00ff4a0 msgr2=0x7f05c0100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:34.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 -- 172.21.15.103:0/1274498673 shutdown_connections 2024-09-16T02:05:34.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.118+0000 7f05c7e00640 1 -- 172.21.15.103:0/1274498673 wait complete. 2024-09-16T02:05:34.136 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T02:05:34.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.293+0000 7fa546600640 1 -- 172.21.15.103:0/4195857261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5401058d0 msgr2=0x7fa54010c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.293+0000 7fa546600640 1 --2- 172.21.15.103:0/4195857261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5401058d0 0x7fa54010c160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa538009ca0 tx=0x7fa53802a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.293+0000 7fa546600640 1 -- 172.21.15.103:0/4195857261 shutdown_connections 2024-09-16T02:05:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.293+0000 7fa546600640 1 --2- 172.21.15.103:0/4195857261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5401058d0 0x7fa54010c160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.293+0000 7fa546600640 1 --2- 172.21.15.103:0/4195857261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540104f10 0x7fa540105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.293+0000 7fa546600640 1 --2- 172.21.15.103:0/4195857261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa540103d20 0x7fa540104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.293+0000 7fa546600640 1 -- 172.21.15.103:0/4195857261 >> 172.21.15.103:0/4195857261 conn(0x7fa5400ff4a0 msgr2=0x7fa5401018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.293+0000 7fa546600640 1 -- 172.21.15.103:0/4195857261 shutdown_connections 2024-09-16T02:05:34.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.293+0000 7fa546600640 1 -- 172.21.15.103:0/4195857261 wait complete. 2024-09-16T02:05:34.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.294+0000 7fa546600640 1 Processor -- start 2024-09-16T02:05:34.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.294+0000 7fa546600640 1 -- start start 2024-09-16T02:05:34.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.294+0000 7fa546600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540103d20 0x7fa540199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.294+0000 7fa546600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa540104f10 0x7fa54019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.294+0000 7fa546600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5401058d0 0x7fa5401a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.294+0000 7fa546600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa54010f9d0 con 0x7fa5401058d0 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.294+0000 7fa546600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa54010f850 con 0x7fa540104f10 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.294+0000 7fa546600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa54010fb50 con 0x7fa540103d20 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa53fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5401058d0 0x7fa5401a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa53fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5401058d0 0x7fa5401a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54550/0 (socket says 172.21.15.103:54550) 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa53fe00640 1 -- 172.21.15.103:0/1413734589 learned_addr learned my addr 172.21.15.103:0/1413734589 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa536a00640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa540104f10 0x7fa54019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa53f400640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540103d20 0x7fa540199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa53fe00640 1 -- 172.21.15.103:0/1413734589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540103d20 msgr2=0x7fa540199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa53fe00640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540103d20 0x7fa540199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa53fe00640 1 -- 172.21.15.103:0/1413734589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa540104f10 msgr2=0x7fa54019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa53fe00640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa540104f10 0x7fa54019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa53fe00640 1 -- 172.21.15.103:0/1413734589 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5401a1c10 con 0x7fa5401058d0 2024-09-16T02:05:34.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa536a00640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa540104f10 0x7fa54019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:34.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.295+0000 7fa53fe00640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5401058d0 0x7fa5401a15a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa538009430 tx=0x7fa5380319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:34.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.296+0000 7fa53cc00640 1 -- 172.21.15.103:0/1413734589 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538003b70 con 0x7fa5401058d0 2024-09-16T02:05:34.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.296+0000 7fa53cc00640 1 -- 172.21.15.103:0/1413734589 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa538003d10 con 0x7fa5401058d0 2024-09-16T02:05:34.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.296+0000 7fa53cc00640 1 -- 172.21.15.103:0/1413734589 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53802fc50 con 0x7fa5401058d0 2024-09-16T02:05:34.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.296+0000 7fa546600640 1 -- 172.21.15.103:0/1413734589 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5401a1ea0 con 0x7fa5401058d0 2024-09-16T02:05:34.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.296+0000 7fa546600640 1 -- 172.21.15.103:0/1413734589 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5401a2360 con 0x7fa5401058d0 2024-09-16T02:05:34.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.297+0000 7fa546600640 1 -- 172.21.15.103:0/1413734589 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa54010b0c0 con 0x7fa5401058d0 2024-09-16T02:05:34.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.299+0000 7fa53cc00640 1 -- 172.21.15.103:0/1413734589 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fa538004040 con 0x7fa5401058d0 2024-09-16T02:05:34.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.301+0000 7fa53cc00640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa5240a3250 0x7fa5240a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:34.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.301+0000 7fa53cc00640 1 -- 172.21.15.103:0/1413734589 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fa5380f0e70 con 0x7fa5401058d0 2024-09-16T02:05:34.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.301+0000 7fa53f400640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa5240a3250 0x7fa5240a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:34.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.301+0000 7fa53f400640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa5240a3250 0x7fa5240a5710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa540104d70 tx=0x7fa530005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:34.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.305+0000 7fa53cc00640 1 -- 172.21.15.103:0/1413734589 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5380bdab0 con 0x7fa5401058d0 2024-09-16T02:05:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.480+0000 7fa546600640 1 -- 172.21.15.103:0/1413734589 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa540062e90 con 0x7fa5401058d0 2024-09-16T02:05:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.480+0000 7fa53cc00640 1 -- 172.21.15.103:0/1413734589 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7fa5380bd3b0 con 0x7fa5401058d0 2024-09-16T02:05:34.483 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 64 2024-09-16T02:05:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 -- 172.21.15.103:0/1413734589 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa5240a3250 msgr2=0x7fa5240a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa5240a3250 0x7fa5240a5710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa540104d70 tx=0x7fa530005f90 comp rx=0 tx=0).stop 2024-09-16T02:05:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 -- 172.21.15.103:0/1413734589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5401058d0 msgr2=0x7fa5401a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5401058d0 0x7fa5401a15a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa538009430 tx=0x7fa5380319f0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 -- 172.21.15.103:0/1413734589 shutdown_connections 2024-09-16T02:05:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa5240a3250 0x7fa5240a5710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5401058d0 0x7fa5401a15a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa540104f10 0x7fa54019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 --2- 172.21.15.103:0/1413734589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540103d20 0x7fa540199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:34.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 -- 172.21.15.103:0/1413734589 >> 172.21.15.103:0/1413734589 conn(0x7fa5400ff4a0 msgr2=0x7fa540108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:34.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 -- 172.21.15.103:0/1413734589 shutdown_connections 2024-09-16T02:05:34.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:34.483+0000 7fa546600640 1 -- 172.21.15.103:0/1413734589 wait complete. 2024-09-16T02:05:34.501 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-16T02:05:34.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:05:34.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:34 smithi103 ceph-mon[30398]: from='client.24994 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:34.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:34 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1855480516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:05:34.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:34 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1274498673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:05:34.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:34 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1413734589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:05:34.878 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:34 smithi136 ceph-mon[35323]: from='client.24994 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:34 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1855480516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:05:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:34 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1274498673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:05:35.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:34 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1413734589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:05:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:34 smithi172 ceph-mon[35340]: from='client.24994 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T02:05:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:34 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1855480516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:05:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:34 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1274498673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:05:35.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:34 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1413734589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:05:35.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.468+0000 7f7d3b200640 1 -- 172.21.15.103:0/1337499738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d34101de0 msgr2=0x7f7d341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.468+0000 7f7d3b200640 1 --2- 172.21.15.103:0/1337499738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d34101de0 0x7f7d341021c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009900 tx=0x7f7d2402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.468+0000 7f7d3b200640 1 -- 172.21.15.103:0/1337499738 shutdown_connections 2024-09-16T02:05:35.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.468+0000 7f7d3b200640 1 --2- 172.21.15.103:0/1337499738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d34106f10 0x7f7d3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.468+0000 7f7d3b200640 1 --2- 172.21.15.103:0/1337499738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d34102790 0x7f7d341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.468+0000 7f7d3b200640 1 --2- 172.21.15.103:0/1337499738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d34101de0 0x7f7d341021c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.468+0000 7f7d3b200640 1 -- 172.21.15.103:0/1337499738 >> 172.21.15.103:0/1337499738 conn(0x7f7d340fd8c0 msgr2=0x7f7d340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:35.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.468+0000 7f7d3b200640 1 -- 172.21.15.103:0/1337499738 shutdown_connections 2024-09-16T02:05:35.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.468+0000 7f7d3b200640 1 -- 172.21.15.103:0/1337499738 wait complete. 2024-09-16T02:05:35.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.469+0000 7f7d3b200640 1 Processor -- start 2024-09-16T02:05:35.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.469+0000 7f7d3b200640 1 -- start start 2024-09-16T02:05:35.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d3b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d34101de0 0x7f7d341984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d3b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d34102790 0x7f7d34198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d3b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d34106f10 0x7f7d3419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d3b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34111f70 con 0x7f7d34101de0 2024-09-16T02:05:35.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d3b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34111df0 con 0x7f7d34102790 2024-09-16T02:05:35.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d3b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d341120f0 con 0x7f7d34106f10 2024-09-16T02:05:35.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d33e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d34101de0 0x7f7d341984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d33e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d34101de0 0x7f7d341984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54572/0 (socket says 172.21.15.103:54572) 2024-09-16T02:05:35.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d33e00640 1 -- 172.21.15.103:0/4200589436 learned_addr learned my addr 172.21.15.103:0/4200589436 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:35.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d39200640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d34106f10 0x7f7d3419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.470+0000 7f7d2b400640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d34102790 0x7f7d34198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d33e00640 1 -- 172.21.15.103:0/4200589436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d34106f10 msgr2=0x7f7d3419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d33e00640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d34106f10 0x7f7d3419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d33e00640 1 -- 172.21.15.103:0/4200589436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d34102790 msgr2=0x7f7d34198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d33e00640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d34102790 0x7f7d34198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d33e00640 1 -- 172.21.15.103:0/4200589436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3419d480 con 0x7f7d34101de0 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d33e00640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d34101de0 0x7f7d341984d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7d24006010 tx=0x7f7d240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d31600640 1 -- 172.21.15.103:0/4200589436 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2403f030 con 0x7f7d34101de0 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d31600640 1 -- 172.21.15.103:0/4200589436 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d24003ce0 con 0x7f7d34101de0 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d3b200640 1 -- 172.21.15.103:0/4200589436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3419d710 con 0x7f7d34101de0 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d31600640 1 -- 172.21.15.103:0/4200589436 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24035070 con 0x7f7d34101de0 2024-09-16T02:05:35.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.471+0000 7f7d3b200640 1 -- 172.21.15.103:0/4200589436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d341a5050 con 0x7f7d34101de0 2024-09-16T02:05:35.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.473+0000 7f7d3b200640 1 -- 172.21.15.103:0/4200589436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d341032c0 con 0x7f7d34101de0 2024-09-16T02:05:35.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.474+0000 7f7d31600640 1 -- 172.21.15.103:0/4200589436 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f7d24042030 con 0x7f7d34101de0 2024-09-16T02:05:35.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.476+0000 7f7d31600640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7d180a32d0 0x7f7d180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:35.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.476+0000 7f7d31600640 1 -- 172.21.15.103:0/4200589436 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f7d240efd50 con 0x7f7d34101de0 2024-09-16T02:05:35.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.476+0000 7f7d2b400640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7d180a32d0 0x7f7d180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:35.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.477+0000 7f7d2b400640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7d180a32d0 0x7f7d180a5790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7d340fdd70 tx=0x7f7d1c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:35.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.480+0000 7f7d31600640 1 -- 172.21.15.103:0/4200589436 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d240bc300 con 0x7f7d34101de0 2024-09-16T02:05:35.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.692+0000 7f7d3b200640 1 -- 172.21.15.103:0/4200589436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d3410a0f0 con 0x7f7d34101de0 2024-09-16T02:05:35.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.692+0000 7f7d31600640 1 -- 172.21.15.103:0/4200589436 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7f7d240c0600 con 0x7f7d34101de0 2024-09-16T02:05:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:35.695 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":24,"btime":"2024-09-16T02:05:32:550231+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:05:31.661572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24506,24505]},"id":1}]} 2024-09-16T02:05:35.695 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 24 2024-09-16T02:05:35.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 -- 172.21.15.103:0/4200589436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7d180a32d0 msgr2=0x7f7d180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7d180a32d0 0x7f7d180a5790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7d340fdd70 tx=0x7f7d1c00b390 comp rx=0 tx=0).stop 2024-09-16T02:05:35.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 -- 172.21.15.103:0/4200589436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d34101de0 msgr2=0x7f7d341984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:35.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d34101de0 0x7f7d341984d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7d24006010 tx=0x7f7d240042f0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 -- 172.21.15.103:0/4200589436 shutdown_connections 2024-09-16T02:05:35.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f7d180a32d0 0x7f7d180a5790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d34106f10 0x7f7d3419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d34102790 0x7f7d34198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 --2- 172.21.15.103:0/4200589436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d34101de0 0x7f7d341984d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 -- 172.21.15.103:0/4200589436 >> 172.21.15.103:0/4200589436 conn(0x7f7d340fd8c0 msgr2=0x7f7d340fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 -- 172.21.15.103:0/4200589436 shutdown_connections 2024-09-16T02:05:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:35.695+0000 7f7d3b200640 1 -- 172.21.15.103:0/4200589436 wait complete. 2024-09-16T02:05:35.701 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:35 smithi103 ceph-mon[30398]: pgmap v344: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-16T02:05:35.701 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:35 smithi103 ceph-mon[30398]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T02:05:36.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:35 smithi136 ceph-mon[35323]: pgmap v344: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-16T02:05:36.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:35 smithi136 ceph-mon[35323]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T02:05:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:35 smithi172 ceph-mon[35340]: pgmap v344: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-16T02:05:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:35 smithi172 ceph-mon[35340]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T02:05:36.453 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-16T02:05:36.453 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-16T02:05:36.454 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T02:05:36.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.608+0000 7f116f200640 1 -- 172.21.15.103:0/800071122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11681058e0 msgr2=0x7f116810c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:36.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.609+0000 7f116f200640 1 --2- 172.21.15.103:0/800071122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11681058e0 0x7f116810c170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1160009ca0 tx=0x7f116002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.609+0000 7f116f200640 1 -- 172.21.15.103:0/800071122 shutdown_connections 2024-09-16T02:05:36.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.609+0000 7f116f200640 1 --2- 172.21.15.103:0/800071122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11681058e0 0x7f116810c170 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.609+0000 7f116f200640 1 --2- 172.21.15.103:0/800071122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1168104f20 0x7f11681053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.609+0000 7f116f200640 1 --2- 172.21.15.103:0/800071122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1168103d30 0x7f1168104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.609+0000 7f116f200640 1 -- 172.21.15.103:0/800071122 >> 172.21.15.103:0/800071122 conn(0x7f11680ff4b0 msgr2=0x7f11681018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:36.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.609+0000 7f116f200640 1 -- 172.21.15.103:0/800071122 shutdown_connections 2024-09-16T02:05:36.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.609+0000 7f116f200640 1 -- 172.21.15.103:0/800071122 wait complete. 2024-09-16T02:05:36.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.610+0000 7f116f200640 1 Processor -- start 2024-09-16T02:05:36.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.610+0000 7f116f200640 1 -- start start 2024-09-16T02:05:36.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.610+0000 7f116f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1168103d30 0x7f116819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:36.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.610+0000 7f116f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1168104f20 0x7f116819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:36.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.610+0000 7f116f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11681058e0 0x7f11681a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:36.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.610+0000 7f116f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f116810f9b0 con 0x7f1168104f20 2024-09-16T02:05:36.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.610+0000 7f116f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f116810f830 con 0x7f11681058e0 2024-09-16T02:05:36.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.610+0000 7f116f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f116810fb30 con 0x7f1168103d30 2024-09-16T02:05:36.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f1167400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1168104f20 0x7f116819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:36.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f1167400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1168104f20 0x7f116819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54586/0 (socket says 172.21.15.103:54586) 2024-09-16T02:05:36.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f1167e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1168103d30 0x7f116819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:36.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f1167400640 1 -- 172.21.15.103:0/2924225308 learned_addr learned my addr 172.21.15.103:0/2924225308 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:36.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f116d200640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11681058e0 0x7f11681a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:36.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f1167400640 1 -- 172.21.15.103:0/2924225308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1168103d30 msgr2=0x7f116819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:36.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f1167400640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1168103d30 0x7f116819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f1167400640 1 -- 172.21.15.103:0/2924225308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11681058e0 msgr2=0x7f11681a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:36.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f1167400640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11681058e0 0x7f11681a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f1167400640 1 -- 172.21.15.103:0/2924225308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11681a1c20 con 0x7f1168104f20 2024-09-16T02:05:36.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.611+0000 7f1167400640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1168104f20 0x7f116819a540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f115400e9f0 tx=0x7f115400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:36.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.613+0000 7f1164c00640 1 -- 172.21.15.103:0/2924225308 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115400cd80 con 0x7f1168104f20 2024-09-16T02:05:36.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.613+0000 7f1164c00640 1 -- 172.21.15.103:0/2924225308 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1154005500 con 0x7f1168104f20 2024-09-16T02:05:36.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.613+0000 7f1164c00640 1 -- 172.21.15.103:0/2924225308 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154010690 con 0x7f1168104f20 2024-09-16T02:05:36.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.613+0000 7f116f200640 1 -- 172.21.15.103:0/2924225308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11681a1f10 con 0x7f1168104f20 2024-09-16T02:05:36.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.613+0000 7f116f200640 1 -- 172.21.15.103:0/2924225308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11681a23d0 con 0x7f1168104f20 2024-09-16T02:05:36.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.614+0000 7f116f200640 1 -- 172.21.15.103:0/2924225308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f116810b0d0 con 0x7f1168104f20 2024-09-16T02:05:36.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.617+0000 7f1164c00640 1 -- 172.21.15.103:0/2924225308 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f11540046c0 con 0x7f1168104f20 2024-09-16T02:05:36.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.618+0000 7f1164c00640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f114c0a3360 0x7f114c0a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:36.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.618+0000 7f1164c00640 1 -- 172.21.15.103:0/2924225308 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f11540cff80 con 0x7f1168104f20 2024-09-16T02:05:36.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.619+0000 7f1167e00640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f114c0a3360 0x7f114c0a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:36.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.619+0000 7f1167e00640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f114c0a3360 0x7f114c0a5820 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1168104d80 tx=0x7f1158005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.624+0000 7f1164c00640 1 -- 172.21.15.103:0/2924225308 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f115409ca60 con 0x7f1168104f20 2024-09-16T02:05:36.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:36 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/4200589436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:36.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.796+0000 7f116f200640 1 -- 172.21.15.103:0/2924225308 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1168062630 con 0x7f114c0a3360 2024-09-16T02:05:36.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.803+0000 7f1164c00640 1 -- 172.21.15.103:0/2924225308 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258899 (secure 0 0 0) 0x7f1168062630 con 0x7f114c0a3360 2024-09-16T02:05:36.805 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:36.810 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-16T02:05:36.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 -- 172.21.15.103:0/2924225308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f114c0a3360 msgr2=0x7f114c0a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f114c0a3360 0x7f114c0a5820 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1168104d80 tx=0x7f1158005f90 comp rx=0 tx=0).stop 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 -- 172.21.15.103:0/2924225308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1168104f20 msgr2=0x7f116819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1168104f20 0x7f116819a540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f115400e9f0 tx=0x7f115400edf0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 -- 172.21.15.103:0/2924225308 shutdown_connections 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f114c0a3360 0x7f114c0a5820 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11681058e0 0x7f11681a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1168104f20 0x7f116819a540 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 --2- 172.21.15.103:0/2924225308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1168103d30 0x7f116819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 -- 172.21.15.103:0/2924225308 >> 172.21.15.103:0/2924225308 conn(0x7f11680ff4b0 msgr2=0x7f11681082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 -- 172.21.15.103:0/2924225308 shutdown_connections 2024-09-16T02:05:36.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:36.807+0000 7f116f200640 1 -- 172.21.15.103:0/2924225308 wait complete. 2024-09-16T02:05:36.825 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-16T02:05:36.478557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655915,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":785,"num_read_kb":513,"num_write":364,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181885,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":791,"ondisk_log_size":791,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370328,"kb_used_data":12688,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571496,"statfs":{"total":1151940427776,"available":1151561211904,"internally_reserved":0,"allocated":12992512,"data_stored":7250940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1153,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":25,"num_write":17,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002242"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547879+0000","last_change":"2024-09-16T02:03:37.050873+0000","last_active":"2024-09-16T02:05:30.547879+0000","last_peered":"2024-09-16T02:05:30.547879+0000","last_clean":"2024-09-16T02:05:30.547879+0000","last_became_active":"2024-09-16T02:03:37.050202+0000","last_became_peered":"2024-09-16T02:03:37.050202+0000","last_unstale":"2024-09-16T02:05:30.547879+0000","last_undegraded":"2024-09-16T02:05:30.547879+0000","last_fullsized":"2024-09-16T02:05:30.547879+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:12.971199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847625+0000","last_change":"2024-09-16T02:03:37.048034+0000","last_active":"2024-09-16T02:05:30.847625+0000","last_peered":"2024-09-16T02:05:30.847625+0000","last_clean":"2024-09-16T02:05:30.847625+0000","last_became_active":"2024-09-16T02:03:37.047216+0000","last_became_peered":"2024-09-16T02:03:37.047216+0000","last_unstale":"2024-09-16T02:05:30.847625+0000","last_undegraded":"2024-09-16T02:05:30.847625+0000","last_fullsized":"2024-09-16T02:05:30.847625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:48.580466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847483+0000","last_change":"2024-09-16T02:03:37.059287+0000","last_active":"2024-09-16T02:05:30.847483+0000","last_peered":"2024-09-16T02:05:30.847483+0000","last_clean":"2024-09-16T02:05:30.847483+0000","last_became_active":"2024-09-16T02:03:37.055270+0000","last_became_peered":"2024-09-16T02:03:37.055270+0000","last_unstale":"2024-09-16T02:05:30.847483+0000","last_undegraded":"2024-09-16T02:05:30.847483+0000","last_fullsized":"2024-09-16T02:05:30.847483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:29.430514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545655+0000","last_change":"2024-09-16T02:03:37.052473+0000","last_active":"2024-09-16T02:05:30.545655+0000","last_peered":"2024-09-16T02:05:30.545655+0000","last_clean":"2024-09-16T02:05:30.545655+0000","last_became_active":"2024-09-16T02:03:37.052015+0000","last_became_peered":"2024-09-16T02:03:37.052015+0000","last_unstale":"2024-09-16T02:05:30.545655+0000","last_undegraded":"2024-09-16T02:05:30.545655+0000","last_fullsized":"2024-09-16T02:05:30.545655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:03.964132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550684+0000","last_change":"2024-09-16T02:05:30.550684+0000","last_active":"2024-09-16T02:05:30.550684+0000","last_peered":"2024-09-16T02:05:30.550684+0000","last_clean":"2024-09-16T02:05:30.550684+0000","last_became_active":"2024-09-16T02:05:30.550245+0000","last_became_peered":"2024-09-16T02:05:30.550245+0000","last_unstale":"2024-09-16T02:05:30.550684+0000","last_undegraded":"2024-09-16T02:05:30.550684+0000","last_fullsized":"2024-09-16T02:05:30.550684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:06.367612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545873+0000","last_change":"2024-09-16T02:03:37.048912+0000","last_active":"2024-09-16T02:05:30.545873+0000","last_peered":"2024-09-16T02:05:30.545873+0000","last_clean":"2024-09-16T02:05:30.545873+0000","last_became_active":"2024-09-16T02:03:37.048577+0000","last_became_peered":"2024-09-16T02:03:37.048577+0000","last_unstale":"2024-09-16T02:05:30.545873+0000","last_undegraded":"2024-09-16T02:05:30.545873+0000","last_fullsized":"2024-09-16T02:05:30.545873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:25.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659580+0000","last_change":"2024-09-16T02:03:37.057427+0000","last_active":"2024-09-16T02:05:30.659580+0000","last_peered":"2024-09-16T02:05:30.659580+0000","last_clean":"2024-09-16T02:05:30.659580+0000","last_became_active":"2024-09-16T02:03:37.057111+0000","last_became_peered":"2024-09-16T02:03:37.057111+0000","last_unstale":"2024-09-16T02:05:30.659580+0000","last_undegraded":"2024-09-16T02:05:30.659580+0000","last_fullsized":"2024-09-16T02:05:30.659580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:44.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926373+0000","last_change":"2024-09-16T02:03:37.051085+0000","last_active":"2024-09-16T02:05:30.926373+0000","last_peered":"2024-09-16T02:05:30.926373+0000","last_clean":"2024-09-16T02:05:30.926373+0000","last_became_active":"2024-09-16T02:03:37.050790+0000","last_became_peered":"2024-09-16T02:03:37.050790+0000","last_unstale":"2024-09-16T02:05:30.926373+0000","last_undegraded":"2024-09-16T02:05:30.926373+0000","last_fullsized":"2024-09-16T02:05:30.926373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:18.620901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549870+0000","last_change":"2024-09-16T02:03:37.054258+0000","last_active":"2024-09-16T02:05:30.549870+0000","last_peered":"2024-09-16T02:05:30.549870+0000","last_clean":"2024-09-16T02:05:30.549870+0000","last_became_active":"2024-09-16T02:03:37.053939+0000","last_became_peered":"2024-09-16T02:03:37.053939+0000","last_unstale":"2024-09-16T02:05:30.549870+0000","last_undegraded":"2024-09-16T02:05:30.549870+0000","last_fullsized":"2024-09-16T02:05:30.549870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:11.076147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.546724+0000","last_change":"2024-09-16T02:03:37.049828+0000","last_active":"2024-09-16T02:05:30.546724+0000","last_peered":"2024-09-16T02:05:30.546724+0000","last_clean":"2024-09-16T02:05:30.546724+0000","last_became_active":"2024-09-16T02:03:37.049577+0000","last_became_peered":"2024-09-16T02:03:37.049577+0000","last_unstale":"2024-09-16T02:05:30.546724+0000","last_undegraded":"2024-09-16T02:05:30.546724+0000","last_fullsized":"2024-09-16T02:05:30.546724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:37.176431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659716+0000","last_change":"2024-09-16T02:03:37.053268+0000","last_active":"2024-09-16T02:05:30.659716+0000","last_peered":"2024-09-16T02:05:30.659716+0000","last_clean":"2024-09-16T02:05:30.659716+0000","last_became_active":"2024-09-16T02:03:37.052933+0000","last_became_peered":"2024-09-16T02:03:37.052933+0000","last_unstale":"2024-09-16T02:05:30.659716+0000","last_undegraded":"2024-09-16T02:05:30.659716+0000","last_fullsized":"2024-09-16T02:05:30.659716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:32.003728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545762+0000","last_change":"2024-09-16T02:03:37.046339+0000","last_active":"2024-09-16T02:05:30.545762+0000","last_peered":"2024-09-16T02:05:30.545762+0000","last_clean":"2024-09-16T02:05:30.545762+0000","last_became_active":"2024-09-16T02:03:37.043262+0000","last_became_peered":"2024-09-16T02:03:37.043262+0000","last_unstale":"2024-09-16T02:05:30.545762+0000","last_undegraded":"2024-09-16T02:05:30.545762+0000","last_fullsized":"2024-09-16T02:05:30.545762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:58.614672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550406+0000","last_change":"2024-09-16T02:05:30.550406+0000","last_active":"2024-09-16T02:05:30.550406+0000","last_peered":"2024-09-16T02:05:30.550406+0000","last_clean":"2024-09-16T02:05:30.550406+0000","last_became_active":"2024-09-16T02:05:30.549945+0000","last_became_peered":"2024-09-16T02:05:30.549945+0000","last_unstale":"2024-09-16T02:05:30.550406+0000","last_undegraded":"2024-09-16T02:05:30.550406+0000","last_fullsized":"2024-09-16T02:05:30.550406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:49.789795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847255+0000","last_change":"2024-09-16T02:03:37.048844+0000","last_active":"2024-09-16T02:05:30.847255+0000","last_peered":"2024-09-16T02:05:30.847255+0000","last_clean":"2024-09-16T02:05:30.847255+0000","last_became_active":"2024-09-16T02:03:37.048522+0000","last_became_peered":"2024-09-16T02:03:37.048522+0000","last_unstale":"2024-09-16T02:05:30.847255+0000","last_undegraded":"2024-09-16T02:05:30.847255+0000","last_fullsized":"2024-09-16T02:05:30.847255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:50.759847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847176+0000","last_change":"2024-09-16T02:03:37.043236+0000","last_active":"2024-09-16T02:05:30.847176+0000","last_peered":"2024-09-16T02:05:30.847176+0000","last_clean":"2024-09-16T02:05:30.847176+0000","last_became_active":"2024-09-16T02:03:37.042967+0000","last_became_peered":"2024-09-16T02:03:37.042967+0000","last_unstale":"2024-09-16T02:05:30.847176+0000","last_undegraded":"2024-09-16T02:05:30.847176+0000","last_fullsized":"2024-09-16T02:05:30.847176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:02.739678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549252+0000","last_change":"2024-09-16T02:03:37.052972+0000","last_active":"2024-09-16T02:05:30.549252+0000","last_peered":"2024-09-16T02:05:30.549252+0000","last_clean":"2024-09-16T02:05:30.549252+0000","last_became_active":"2024-09-16T02:03:37.052665+0000","last_became_peered":"2024-09-16T02:03:37.052665+0000","last_unstale":"2024-09-16T02:05:30.549252+0000","last_undegraded":"2024-09-16T02:05:30.549252+0000","last_fullsized":"2024-09-16T02:05:30.549252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:59.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547979+0000","last_change":"2024-09-16T02:03:37.051203+0000","last_active":"2024-09-16T02:05:30.547979+0000","last_peered":"2024-09-16T02:05:30.547979+0000","last_clean":"2024-09-16T02:05:30.547979+0000","last_became_active":"2024-09-16T02:03:37.049828+0000","last_became_peered":"2024-09-16T02:03:37.049828+0000","last_unstale":"2024-09-16T02:05:30.547979+0000","last_undegraded":"2024-09-16T02:05:30.547979+0000","last_fullsized":"2024-09-16T02:05:30.547979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:47.566594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548575+0000","last_change":"2024-09-16T02:03:37.056903+0000","last_active":"2024-09-16T02:05:30.548575+0000","last_peered":"2024-09-16T02:05:30.548575+0000","last_clean":"2024-09-16T02:05:30.548575+0000","last_became_active":"2024-09-16T02:03:37.056579+0000","last_became_peered":"2024-09-16T02:03:37.056579+0000","last_unstale":"2024-09-16T02:05:30.548575+0000","last_undegraded":"2024-09-16T02:05:30.548575+0000","last_fullsized":"2024-09-16T02:05:30.548575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:23.652294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544817+0000","last_change":"2024-09-16T02:03:37.054413+0000","last_active":"2024-09-16T02:05:30.544817+0000","last_peered":"2024-09-16T02:05:30.544817+0000","last_clean":"2024-09-16T02:05:30.544817+0000","last_became_active":"2024-09-16T02:03:37.054040+0000","last_became_peered":"2024-09-16T02:03:37.054040+0000","last_unstale":"2024-09-16T02:05:30.544817+0000","last_undegraded":"2024-09-16T02:05:30.544817+0000","last_fullsized":"2024-09-16T02:05:30.544817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:04.517313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547502+0000","last_change":"2024-09-16T02:03:37.053454+0000","last_active":"2024-09-16T02:05:30.547502+0000","last_peered":"2024-09-16T02:05:30.547502+0000","last_clean":"2024-09-16T02:05:30.547502+0000","last_became_active":"2024-09-16T02:03:37.053123+0000","last_became_peered":"2024-09-16T02:03:37.053123+0000","last_unstale":"2024-09-16T02:05:30.547502+0000","last_undegraded":"2024-09-16T02:05:30.547502+0000","last_fullsized":"2024-09-16T02:05:30.547502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:52.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543738+0000","last_change":"2024-09-16T02:03:37.042772+0000","last_active":"2024-09-16T02:05:30.543738+0000","last_peered":"2024-09-16T02:05:30.543738+0000","last_clean":"2024-09-16T02:05:30.543738+0000","last_became_active":"2024-09-16T02:03:37.041832+0000","last_became_peered":"2024-09-16T02:03:37.041832+0000","last_unstale":"2024-09-16T02:05:30.543738+0000","last_undegraded":"2024-09-16T02:05:30.543738+0000","last_fullsized":"2024-09-16T02:05:30.543738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:54.436291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549491+0000","last_change":"2024-09-16T02:03:37.058484+0000","last_active":"2024-09-16T02:05:30.549491+0000","last_peered":"2024-09-16T02:05:30.549491+0000","last_clean":"2024-09-16T02:05:30.549491+0000","last_became_active":"2024-09-16T02:03:37.058047+0000","last_became_peered":"2024-09-16T02:03:37.058047+0000","last_unstale":"2024-09-16T02:05:30.549491+0000","last_undegraded":"2024-09-16T02:05:30.549491+0000","last_fullsized":"2024-09-16T02:05:30.549491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:23.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926399+0000","last_change":"2024-09-16T02:03:37.058576+0000","last_active":"2024-09-16T02:05:30.926399+0000","last_peered":"2024-09-16T02:05:30.926399+0000","last_clean":"2024-09-16T02:05:30.926399+0000","last_became_active":"2024-09-16T02:03:37.057624+0000","last_became_peered":"2024-09-16T02:03:37.057624+0000","last_unstale":"2024-09-16T02:05:30.926399+0000","last_undegraded":"2024-09-16T02:05:30.926399+0000","last_fullsized":"2024-09-16T02:05:30.926399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:21.929461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549455+0000","last_change":"2024-09-16T02:03:37.050889+0000","last_active":"2024-09-16T02:05:30.549455+0000","last_peered":"2024-09-16T02:05:30.549455+0000","last_clean":"2024-09-16T02:05:30.549455+0000","last_became_active":"2024-09-16T02:03:37.050535+0000","last_became_peered":"2024-09-16T02:03:37.050535+0000","last_unstale":"2024-09-16T02:05:30.549455+0000","last_undegraded":"2024-09-16T02:05:30.549455+0000","last_fullsized":"2024-09-16T02:05:30.549455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:46.955962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926844+0000","last_change":"2024-09-16T02:03:37.053865+0000","last_active":"2024-09-16T02:05:30.926844+0000","last_peered":"2024-09-16T02:05:30.926844+0000","last_clean":"2024-09-16T02:05:30.926844+0000","last_became_active":"2024-09-16T02:03:37.053563+0000","last_became_peered":"2024-09-16T02:03:37.053563+0000","last_unstale":"2024-09-16T02:05:30.926844+0000","last_undegraded":"2024-09-16T02:05:30.926844+0000","last_fullsized":"2024-09-16T02:05:30.926844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:24.109471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545076+0000","last_change":"2024-09-16T02:03:37.054681+0000","last_active":"2024-09-16T02:05:30.545076+0000","last_peered":"2024-09-16T02:05:30.545076+0000","last_clean":"2024-09-16T02:05:30.545076+0000","last_became_active":"2024-09-16T02:03:37.054324+0000","last_became_peered":"2024-09-16T02:03:37.054324+0000","last_unstale":"2024-09-16T02:05:30.545076+0000","last_undegraded":"2024-09-16T02:05:30.545076+0000","last_fullsized":"2024-09-16T02:05:30.545076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:20.472520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547733+0000","last_change":"2024-09-16T02:03:37.047713+0000","last_active":"2024-09-16T02:05:30.547733+0000","last_peered":"2024-09-16T02:05:30.547733+0000","last_clean":"2024-09-16T02:05:30.547733+0000","last_became_active":"2024-09-16T02:03:37.047353+0000","last_became_peered":"2024-09-16T02:03:37.047353+0000","last_unstale":"2024-09-16T02:05:30.547733+0000","last_undegraded":"2024-09-16T02:05:30.547733+0000","last_fullsized":"2024-09-16T02:05:30.547733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:05.868909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543731+0000","last_change":"2024-09-16T02:03:37.038931+0000","last_active":"2024-09-16T02:05:30.543731+0000","last_peered":"2024-09-16T02:05:30.543731+0000","last_clean":"2024-09-16T02:05:30.543731+0000","last_became_active":"2024-09-16T02:03:37.037234+0000","last_became_peered":"2024-09-16T02:03:37.037234+0000","last_unstale":"2024-09-16T02:05:30.543731+0000","last_undegraded":"2024-09-16T02:05:30.543731+0000","last_fullsized":"2024-09-16T02:05:30.543731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:54.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847518+0000","last_change":"2024-09-16T02:04:29.634327+0000","last_active":"2024-09-16T02:05:30.847518+0000","last_peered":"2024-09-16T02:05:30.847518+0000","last_clean":"2024-09-16T02:05:30.847518+0000","last_became_active":"2024-09-16T02:04:29.633710+0000","last_became_peered":"2024-09-16T02:04:29.633710+0000","last_unstale":"2024-09-16T02:05:30.847518+0000","last_undegraded":"2024-09-16T02:05:30.847518+0000","last_fullsized":"2024-09-16T02:05:30.847518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:55:03.574092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544503+0000","last_change":"2024-09-16T02:03:37.057688+0000","last_active":"2024-09-16T02:05:30.544503+0000","last_peered":"2024-09-16T02:05:30.544503+0000","last_clean":"2024-09-16T02:05:30.544503+0000","last_became_active":"2024-09-16T02:03:37.057347+0000","last_became_peered":"2024-09-16T02:03:37.057347+0000","last_unstale":"2024-09-16T02:05:30.544503+0000","last_undegraded":"2024-09-16T02:05:30.544503+0000","last_fullsized":"2024-09-16T02:05:30.544503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:30.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544749+0000","last_change":"2024-09-16T02:04:29.640455+0000","last_active":"2024-09-16T02:05:30.544749+0000","last_peered":"2024-09-16T02:05:30.544749+0000","last_clean":"2024-09-16T02:05:30.544749+0000","last_became_active":"2024-09-16T02:04:29.640083+0000","last_became_peered":"2024-09-16T02:04:29.640083+0000","last_unstale":"2024-09-16T02:05:30.544749+0000","last_undegraded":"2024-09-16T02:05:30.544749+0000","last_fullsized":"2024-09-16T02:05:30.544749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:46.191484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547527+0000","last_change":"2024-09-16T02:03:37.043596+0000","last_active":"2024-09-16T02:05:30.547527+0000","last_peered":"2024-09-16T02:05:30.547527+0000","last_clean":"2024-09-16T02:05:30.547527+0000","last_became_active":"2024-09-16T02:03:37.043261+0000","last_became_peered":"2024-09-16T02:03:37.043261+0000","last_unstale":"2024-09-16T02:05:30.547527+0000","last_undegraded":"2024-09-16T02:05:30.547527+0000","last_fullsized":"2024-09-16T02:05:30.547527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:03.833126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545258+0000","last_change":"2024-09-16T02:03:37.043018+0000","last_active":"2024-09-16T02:05:30.545258+0000","last_peered":"2024-09-16T02:05:30.545258+0000","last_clean":"2024-09-16T02:05:30.545258+0000","last_became_active":"2024-09-16T02:03:37.042437+0000","last_became_peered":"2024-09-16T02:03:37.042437+0000","last_unstale":"2024-09-16T02:05:30.545258+0000","last_undegraded":"2024-09-16T02:05:30.545258+0000","last_fullsized":"2024-09-16T02:05:30.545258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.193852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'7","reported_seq":153,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.739530+0000","last_change":"2024-09-16T02:04:29.645481+0000","last_active":"2024-09-16T02:05:30.739530+0000","last_peered":"2024-09-16T02:05:30.739530+0000","last_clean":"2024-09-16T02:05:30.739530+0000","last_became_active":"2024-09-16T02:04:29.640089+0000","last_became_peered":"2024-09-16T02:04:29.640089+0000","last_unstale":"2024-09-16T02:05:30.739530+0000","last_undegraded":"2024-09-16T02:05:30.739530+0000","last_fullsized":"2024-09-16T02:05:30.739530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:02.431443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549918+0000","last_change":"2024-09-16T02:03:37.052664+0000","last_active":"2024-09-16T02:05:30.549918+0000","last_peered":"2024-09-16T02:05:30.549918+0000","last_clean":"2024-09-16T02:05:30.549918+0000","last_became_active":"2024-09-16T02:03:37.051310+0000","last_became_peered":"2024-09-16T02:03:37.051310+0000","last_unstale":"2024-09-16T02:05:30.549918+0000","last_undegraded":"2024-09-16T02:05:30.549918+0000","last_fullsized":"2024-09-16T02:05:30.549918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:57.979053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547503+0000","last_change":"2024-09-16T02:04:29.654377+0000","last_active":"2024-09-16T02:05:30.547503+0000","last_peered":"2024-09-16T02:05:30.547503+0000","last_clean":"2024-09-16T02:05:30.547503+0000","last_became_active":"2024-09-16T02:04:29.635187+0000","last_became_peered":"2024-09-16T02:04:29.635187+0000","last_unstale":"2024-09-16T02:05:30.547503+0000","last_undegraded":"2024-09-16T02:05:30.547503+0000","last_fullsized":"2024-09-16T02:05:30.547503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:28.474841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,6],"acting":[11,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543512+0000","last_change":"2024-09-16T02:03:34.745325+0000","last_active":"2024-09-16T02:05:30.543512+0000","last_peered":"2024-09-16T02:05:30.543512+0000","last_clean":"2024-09-16T02:05:30.543512+0000","last_became_active":"2024-09-16T02:03:34.744730+0000","last_became_peered":"2024-09-16T02:03:34.744730+0000","last_unstale":"2024-09-16T02:05:30.543512+0000","last_undegraded":"2024-09-16T02:05:30.543512+0000","last_fullsized":"2024-09-16T02:05:30.543512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926402+0000","last_change":"2024-09-16T02:03:37.058973+0000","last_active":"2024-09-16T02:05:30.926402+0000","last_peered":"2024-09-16T02:05:30.926402+0000","last_clean":"2024-09-16T02:05:30.926402+0000","last_became_active":"2024-09-16T02:03:37.058685+0000","last_became_peered":"2024-09-16T02:03:37.058685+0000","last_unstale":"2024-09-16T02:05:30.926402+0000","last_undegraded":"2024-09-16T02:05:30.926402+0000","last_fullsized":"2024-09-16T02:05:30.926402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:46.030440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549707+0000","last_change":"2024-09-16T02:03:34.737842+0000","last_active":"2024-09-16T02:05:30.549707+0000","last_peered":"2024-09-16T02:05:30.549707+0000","last_clean":"2024-09-16T02:05:30.549707+0000","last_became_active":"2024-09-16T02:03:34.737391+0000","last_became_peered":"2024-09-16T02:03:34.737391+0000","last_unstale":"2024-09-16T02:05:30.549707+0000","last_undegraded":"2024-09-16T02:05:30.549707+0000","last_fullsized":"2024-09-16T02:05:30.549707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:39.214180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548437+0000","last_change":"2024-09-16T02:03:37.049705+0000","last_active":"2024-09-16T02:05:30.548437+0000","last_peered":"2024-09-16T02:05:30.548437+0000","last_clean":"2024-09-16T02:05:30.548437+0000","last_became_active":"2024-09-16T02:03:37.049392+0000","last_became_peered":"2024-09-16T02:03:37.049392+0000","last_unstale":"2024-09-16T02:05:30.548437+0000","last_undegraded":"2024-09-16T02:05:30.548437+0000","last_fullsized":"2024-09-16T02:05:30.548437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.932391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:31.740755+0000","last_change":"2024-09-16T02:03:34.742268+0000","last_active":"2024-09-16T02:05:31.740755+0000","last_peered":"2024-09-16T02:05:31.740755+0000","last_clean":"2024-09-16T02:05:31.740755+0000","last_became_active":"2024-09-16T02:03:34.741934+0000","last_became_peered":"2024-09-16T02:03:34.741934+0000","last_unstale":"2024-09-16T02:05:31.740755+0000","last_undegraded":"2024-09-16T02:05:31.740755+0000","last_fullsized":"2024-09-16T02:05:31.740755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:30.301552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549786+0000","last_change":"2024-09-16T02:03:37.059119+0000","last_active":"2024-09-16T02:05:30.549786+0000","last_peered":"2024-09-16T02:05:30.549786+0000","last_clean":"2024-09-16T02:05:30.549786+0000","last_became_active":"2024-09-16T02:03:37.058738+0000","last_became_peered":"2024-09-16T02:03:37.058738+0000","last_unstale":"2024-09-16T02:05:30.549786+0000","last_undegraded":"2024-09-16T02:05:30.549786+0000","last_fullsized":"2024-09-16T02:05:30.549786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.496053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659981+0000","last_change":"2024-09-16T02:04:29.642379+0000","last_active":"2024-09-16T02:05:30.659981+0000","last_peered":"2024-09-16T02:05:30.659981+0000","last_clean":"2024-09-16T02:05:30.659981+0000","last_became_active":"2024-09-16T02:04:29.638587+0000","last_became_peered":"2024-09-16T02:04:29.638587+0000","last_unstale":"2024-09-16T02:05:30.659981+0000","last_undegraded":"2024-09-16T02:05:30.659981+0000","last_fullsized":"2024-09-16T02:05:30.659981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:50.268223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847445+0000","last_change":"2024-09-16T02:03:37.050349+0000","last_active":"2024-09-16T02:05:30.847445+0000","last_peered":"2024-09-16T02:05:30.847445+0000","last_clean":"2024-09-16T02:05:30.847445+0000","last_became_active":"2024-09-16T02:03:37.049978+0000","last_became_peered":"2024-09-16T02:03:37.049978+0000","last_unstale":"2024-09-16T02:05:30.847445+0000","last_undegraded":"2024-09-16T02:05:30.847445+0000","last_fullsized":"2024-09-16T02:05:30.847445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:45.119683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550509+0000","last_change":"2024-09-16T02:03:34.740246+0000","last_active":"2024-09-16T02:05:30.550509+0000","last_peered":"2024-09-16T02:05:30.550509+0000","last_clean":"2024-09-16T02:05:30.550509+0000","last_became_active":"2024-09-16T02:03:34.739908+0000","last_became_peered":"2024-09-16T02:03:34.739908+0000","last_unstale":"2024-09-16T02:05:30.550509+0000","last_undegraded":"2024-09-16T02:05:30.550509+0000","last_fullsized":"2024-09-16T02:05:30.550509+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:35.299733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549580+0000","last_change":"2024-09-16T02:03:37.056562+0000","last_active":"2024-09-16T02:05:30.549580+0000","last_peered":"2024-09-16T02:05:30.549580+0000","last_clean":"2024-09-16T02:05:30.549580+0000","last_became_active":"2024-09-16T02:03:37.056226+0000","last_became_peered":"2024-09-16T02:03:37.056226+0000","last_unstale":"2024-09-16T02:05:30.549580+0000","last_undegraded":"2024-09-16T02:05:30.549580+0000","last_fullsized":"2024-09-16T02:05:30.549580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.741538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":279,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:32.742423+0000","last_change":"2024-09-16T02:03:34.736403+0000","last_active":"2024-09-16T02:05:32.742423+0000","last_peered":"2024-09-16T02:05:32.742423+0000","last_clean":"2024-09-16T02:05:32.742423+0000","last_became_active":"2024-09-16T02:03:34.736074+0000","last_became_peered":"2024-09-16T02:03:34.736074+0000","last_unstale":"2024-09-16T02:05:32.742423+0000","last_undegraded":"2024-09-16T02:05:32.742423+0000","last_fullsized":"2024-09-16T02:05:32.742423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:34.769862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60324,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":169,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545616+0000","last_change":"2024-09-16T02:03:37.046803+0000","last_active":"2024-09-16T02:05:30.545616+0000","last_peered":"2024-09-16T02:05:30.545616+0000","last_clean":"2024-09-16T02:05:30.545616+0000","last_became_active":"2024-09-16T02:03:37.046459+0000","last_became_peered":"2024-09-16T02:03:37.046459+0000","last_unstale":"2024-09-16T02:05:30.545616+0000","last_undegraded":"2024-09-16T02:05:30.545616+0000","last_fullsized":"2024-09-16T02:05:30.545616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:55:38.460795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544547+0000","last_change":"2024-09-16T02:03:34.739016+0000","last_active":"2024-09-16T02:05:30.544547+0000","last_peered":"2024-09-16T02:05:30.544547+0000","last_clean":"2024-09-16T02:05:30.544547+0000","last_became_active":"2024-09-16T02:03:34.738583+0000","last_became_peered":"2024-09-16T02:03:34.738583+0000","last_unstale":"2024-09-16T02:05:30.544547+0000","last_undegraded":"2024-09-16T02:05:30.544547+0000","last_fullsized":"2024-09-16T02:05:30.544547+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:08.255832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543487+0000","last_change":"2024-09-16T02:03:37.053353+0000","last_active":"2024-09-16T02:05:30.543487+0000","last_peered":"2024-09-16T02:05:30.543487+0000","last_clean":"2024-09-16T02:05:30.543487+0000","last_became_active":"2024-09-16T02:03:37.051140+0000","last_became_peered":"2024-09-16T02:03:37.051140+0000","last_unstale":"2024-09-16T02:05:30.543487+0000","last_undegraded":"2024-09-16T02:05:30.543487+0000","last_fullsized":"2024-09-16T02:05:30.543487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:34.636338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549839+0000","last_change":"2024-09-16T02:03:34.742111+0000","last_active":"2024-09-16T02:05:30.549839+0000","last_peered":"2024-09-16T02:05:30.549839+0000","last_clean":"2024-09-16T02:05:30.549839+0000","last_became_active":"2024-09-16T02:03:34.741816+0000","last_became_peered":"2024-09-16T02:03:34.741816+0000","last_unstale":"2024-09-16T02:05:30.549839+0000","last_undegraded":"2024-09-16T02:05:30.549839+0000","last_fullsized":"2024-09-16T02:05:30.549839+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:36.861995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659571+0000","last_change":"2024-09-16T02:03:37.047152+0000","last_active":"2024-09-16T02:05:30.659571+0000","last_peered":"2024-09-16T02:05:30.659571+0000","last_clean":"2024-09-16T02:05:30.659571+0000","last_became_active":"2024-09-16T02:03:37.046675+0000","last_became_peered":"2024-09-16T02:03:37.046675+0000","last_unstale":"2024-09-16T02:05:30.659571+0000","last_undegraded":"2024-09-16T02:05:30.659571+0000","last_fullsized":"2024-09-16T02:05:30.659571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:58.236519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":142,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:31.267665+0000","last_change":"2024-09-16T02:03:34.738487+0000","last_active":"2024-09-16T02:05:31.267665+0000","last_peered":"2024-09-16T02:05:31.267665+0000","last_clean":"2024-09-16T02:05:31.267665+0000","last_became_active":"2024-09-16T02:03:34.737061+0000","last_became_peered":"2024-09-16T02:03:34.737061+0000","last_unstale":"2024-09-16T02:05:31.267665+0000","last_undegraded":"2024-09-16T02:05:31.267665+0000","last_fullsized":"2024-09-16T02:05:31.267665+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:24.231640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1433,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659353+0000","last_change":"2024-09-16T02:03:37.044260+0000","last_active":"2024-09-16T02:05:30.659353+0000","last_peered":"2024-09-16T02:05:30.659353+0000","last_clean":"2024-09-16T02:05:30.659353+0000","last_became_active":"2024-09-16T02:03:37.043932+0000","last_became_peered":"2024-09-16T02:03:37.043932+0000","last_unstale":"2024-09-16T02:05:30.659353+0000","last_undegraded":"2024-09-16T02:05:30.659353+0000","last_fullsized":"2024-09-16T02:05:30.659353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:55.718155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547422+0000","last_change":"2024-09-16T02:04:29.651235+0000","last_active":"2024-09-16T02:05:30.547422+0000","last_peered":"2024-09-16T02:05:30.547422+0000","last_clean":"2024-09-16T02:05:30.547422+0000","last_became_active":"2024-09-16T02:04:29.633999+0000","last_became_peered":"2024-09-16T02:04:29.633999+0000","last_unstale":"2024-09-16T02:05:30.547422+0000","last_undegraded":"2024-09-16T02:05:30.547422+0000","last_fullsized":"2024-09-16T02:05:30.547422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.616990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549533+0000","last_change":"2024-09-16T02:03:37.053040+0000","last_active":"2024-09-16T02:05:30.549533+0000","last_peered":"2024-09-16T02:05:30.549533+0000","last_clean":"2024-09-16T02:05:30.549533+0000","last_became_active":"2024-09-16T02:03:37.052185+0000","last_became_peered":"2024-09-16T02:03:37.052185+0000","last_unstale":"2024-09-16T02:05:30.549533+0000","last_undegraded":"2024-09-16T02:05:30.549533+0000","last_fullsized":"2024-09-16T02:05:30.549533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:27.676678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'3","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545839+0000","last_change":"2024-09-16T02:04:29.651376+0000","last_active":"2024-09-16T02:05:30.545839+0000","last_peered":"2024-09-16T02:05:30.545839+0000","last_clean":"2024-09-16T02:05:30.545839+0000","last_became_active":"2024-09-16T02:04:29.640260+0000","last_became_peered":"2024-09-16T02:04:29.640260+0000","last_unstale":"2024-09-16T02:05:30.545839+0000","last_undegraded":"2024-09-16T02:05:30.545839+0000","last_fullsized":"2024-09-16T02:05:30.545839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:39.426183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.661732+0000","last_change":"2024-09-16T02:05:30.661732+0000","last_active":"2024-09-16T02:05:30.661732+0000","last_peered":"2024-09-16T02:05:30.661732+0000","last_clean":"2024-09-16T02:05:30.661732+0000","last_became_active":"2024-09-16T02:05:30.661261+0000","last_became_peered":"2024-09-16T02:05:30.661261+0000","last_unstale":"2024-09-16T02:05:30.661732+0000","last_undegraded":"2024-09-16T02:05:30.661732+0000","last_fullsized":"2024-09-16T02:05:30.661732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:10.510530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543828+0000","last_change":"2024-09-16T02:03:34.738516+0000","last_active":"2024-09-16T02:05:30.543828+0000","last_peered":"2024-09-16T02:05:30.543828+0000","last_clean":"2024-09-16T02:05:30.543828+0000","last_became_active":"2024-09-16T02:03:34.737701+0000","last_became_peered":"2024-09-16T02:03:34.737701+0000","last_unstale":"2024-09-16T02:05:30.543828+0000","last_undegraded":"2024-09-16T02:05:30.543828+0000","last_fullsized":"2024-09-16T02:05:30.543828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:42.101865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659922+0000","last_change":"2024-09-16T02:03:37.046820+0000","last_active":"2024-09-16T02:05:30.659922+0000","last_peered":"2024-09-16T02:05:30.659922+0000","last_clean":"2024-09-16T02:05:30.659922+0000","last_became_active":"2024-09-16T02:03:37.046082+0000","last_became_peered":"2024-09-16T02:03:37.046082+0000","last_unstale":"2024-09-16T02:05:30.659922+0000","last_undegraded":"2024-09-16T02:05:30.659922+0000","last_fullsized":"2024-09-16T02:05:30.659922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:21.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550670+0000","last_change":"2024-09-16T02:03:34.741908+0000","last_active":"2024-09-16T02:05:30.550670+0000","last_peered":"2024-09-16T02:05:30.550670+0000","last_clean":"2024-09-16T02:05:30.550670+0000","last_became_active":"2024-09-16T02:03:34.739826+0000","last_became_peered":"2024-09-16T02:03:34.739826+0000","last_unstale":"2024-09-16T02:05:30.550670+0000","last_undegraded":"2024-09-16T02:05:30.550670+0000","last_fullsized":"2024-09-16T02:05:30.550670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:04.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659551+0000","last_change":"2024-09-16T02:03:37.056459+0000","last_active":"2024-09-16T02:05:30.659551+0000","last_peered":"2024-09-16T02:05:30.659551+0000","last_clean":"2024-09-16T02:05:30.659551+0000","last_became_active":"2024-09-16T02:03:37.055902+0000","last_became_peered":"2024-09-16T02:03:37.055902+0000","last_unstale":"2024-09-16T02:05:30.659551+0000","last_undegraded":"2024-09-16T02:05:30.659551+0000","last_fullsized":"2024-09-16T02:05:30.659551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:48:30.815654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544898+0000","last_change":"2024-09-16T02:03:34.741207+0000","last_active":"2024-09-16T02:05:30.544898+0000","last_peered":"2024-09-16T02:05:30.544898+0000","last_clean":"2024-09-16T02:05:30.544898+0000","last_became_active":"2024-09-16T02:03:34.740523+0000","last_became_peered":"2024-09-16T02:03:34.740523+0000","last_unstale":"2024-09-16T02:05:30.544898+0000","last_undegraded":"2024-09-16T02:05:30.544898+0000","last_fullsized":"2024-09-16T02:05:30.544898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:36.199332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659815+0000","last_change":"2024-09-16T02:03:37.052792+0000","last_active":"2024-09-16T02:05:30.659815+0000","last_peered":"2024-09-16T02:05:30.659815+0000","last_clean":"2024-09-16T02:05:30.659815+0000","last_became_active":"2024-09-16T02:03:37.052301+0000","last_became_peered":"2024-09-16T02:03:37.052301+0000","last_unstale":"2024-09-16T02:05:30.659815+0000","last_undegraded":"2024-09-16T02:05:30.659815+0000","last_fullsized":"2024-09-16T02:05:30.659815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:55.197481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":338,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659545+0000","last_change":"2024-09-16T02:02:21.095893+0000","last_active":"2024-09-16T02:05:30.659545+0000","last_peered":"2024-09-16T02:05:30.659545+0000","last_clean":"2024-09-16T02:05:30.659545+0000","last_became_active":"2024-09-16T02:02:21.095395+0000","last_became_peered":"2024-09-16T02:02:21.095395+0000","last_unstale":"2024-09-16T02:05:30.659545+0000","last_undegraded":"2024-09-16T02:05:30.659545+0000","last_fullsized":"2024-09-16T02:05:30.659545+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_clean_scrub_stamp":"2024-09-16T01:58:50.954254+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:40.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545471+0000","last_change":"2024-09-16T02:03:34.746154+0000","last_active":"2024-09-16T02:05:30.545471+0000","last_peered":"2024-09-16T02:05:30.545471+0000","last_clean":"2024-09-16T02:05:30.545471+0000","last_became_active":"2024-09-16T02:03:34.745646+0000","last_became_peered":"2024-09-16T02:03:34.745646+0000","last_unstale":"2024-09-16T02:05:30.545471+0000","last_undegraded":"2024-09-16T02:05:30.545471+0000","last_fullsized":"2024-09-16T02:05:30.545471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:06.790000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926706+0000","last_change":"2024-09-16T02:03:37.056638+0000","last_active":"2024-09-16T02:05:30.926706+0000","last_peered":"2024-09-16T02:05:30.926706+0000","last_clean":"2024-09-16T02:05:30.926706+0000","last_became_active":"2024-09-16T02:03:37.056043+0000","last_became_peered":"2024-09-16T02:03:37.056043+0000","last_unstale":"2024-09-16T02:05:30.926706+0000","last_undegraded":"2024-09-16T02:05:30.926706+0000","last_fullsized":"2024-09-16T02:05:30.926706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:35:48.353380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659548+0000","last_change":"2024-09-16T02:04:29.647682+0000","last_active":"2024-09-16T02:05:30.659548+0000","last_peered":"2024-09-16T02:05:30.659548+0000","last_clean":"2024-09-16T02:05:30.659548+0000","last_became_active":"2024-09-16T02:04:29.640065+0000","last_became_peered":"2024-09-16T02:04:29.640065+0000","last_unstale":"2024-09-16T02:05:30.659548+0000","last_undegraded":"2024-09-16T02:05:30.659548+0000","last_fullsized":"2024-09-16T02:05:30.659548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:53.765121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543418+0000","last_change":"2024-09-16T02:03:37.050895+0000","last_active":"2024-09-16T02:05:30.543418+0000","last_peered":"2024-09-16T02:05:30.543418+0000","last_clean":"2024-09-16T02:05:30.543418+0000","last_became_active":"2024-09-16T02:03:37.050559+0000","last_became_peered":"2024-09-16T02:03:37.050559+0000","last_unstale":"2024-09-16T02:05:30.543418+0000","last_undegraded":"2024-09-16T02:05:30.543418+0000","last_fullsized":"2024-09-16T02:05:30.543418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:20.067954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:34.271880+0000","last_change":"2024-09-16T02:03:34.740286+0000","last_active":"2024-09-16T02:05:34.271880+0000","last_peered":"2024-09-16T02:05:34.271880+0000","last_clean":"2024-09-16T02:05:34.271880+0000","last_became_active":"2024-09-16T02:03:34.739892+0000","last_became_peered":"2024-09-16T02:03:34.739892+0000","last_unstale":"2024-09-16T02:05:34.271880+0000","last_undegraded":"2024-09-16T02:05:34.271880+0000","last_fullsized":"2024-09-16T02:05:34.271880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:59:14.809991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549598+0000","last_change":"2024-09-16T02:03:37.042207+0000","last_active":"2024-09-16T02:05:30.549598+0000","last_peered":"2024-09-16T02:05:30.549598+0000","last_clean":"2024-09-16T02:05:30.549598+0000","last_became_active":"2024-09-16T02:03:37.041283+0000","last_became_peered":"2024-09-16T02:03:37.041283+0000","last_unstale":"2024-09-16T02:05:30.549598+0000","last_undegraded":"2024-09-16T02:05:30.549598+0000","last_fullsized":"2024-09-16T02:05:30.549598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:45.569721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659831+0000","last_change":"2024-09-16T02:03:34.737834+0000","last_active":"2024-09-16T02:05:30.659831+0000","last_peered":"2024-09-16T02:05:30.659831+0000","last_clean":"2024-09-16T02:05:30.659831+0000","last_became_active":"2024-09-16T02:03:34.736706+0000","last_became_peered":"2024-09-16T02:03:34.736706+0000","last_unstale":"2024-09-16T02:05:30.659831+0000","last_undegraded":"2024-09-16T02:05:30.659831+0000","last_fullsized":"2024-09-16T02:05:30.659831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:27.037351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.546182+0000","last_change":"2024-09-16T02:03:37.060888+0000","last_active":"2024-09-16T02:05:30.546182+0000","last_peered":"2024-09-16T02:05:30.546182+0000","last_clean":"2024-09-16T02:05:30.546182+0000","last_became_active":"2024-09-16T02:03:37.060458+0000","last_became_peered":"2024-09-16T02:03:37.060458+0000","last_unstale":"2024-09-16T02:05:30.546182+0000","last_undegraded":"2024-09-16T02:05:30.546182+0000","last_fullsized":"2024-09-16T02:05:30.546182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:37.425376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548446+0000","last_change":"2024-09-16T02:03:34.742298+0000","last_active":"2024-09-16T02:05:30.548446+0000","last_peered":"2024-09-16T02:05:30.548446+0000","last_clean":"2024-09-16T02:05:30.548446+0000","last_became_active":"2024-09-16T02:03:34.741960+0000","last_became_peered":"2024-09-16T02:03:34.741960+0000","last_unstale":"2024-09-16T02:05:30.548446+0000","last_undegraded":"2024-09-16T02:05:30.548446+0000","last_fullsized":"2024-09-16T02:05:30.548446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:12.832207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545922+0000","last_change":"2024-09-16T02:03:37.060553+0000","last_active":"2024-09-16T02:05:30.545922+0000","last_peered":"2024-09-16T02:05:30.545922+0000","last_clean":"2024-09-16T02:05:30.545922+0000","last_became_active":"2024-09-16T02:03:37.059961+0000","last_became_peered":"2024-09-16T02:03:37.059961+0000","last_unstale":"2024-09-16T02:05:30.545922+0000","last_undegraded":"2024-09-16T02:05:30.545922+0000","last_fullsized":"2024-09-16T02:05:30.545922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:22.904190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545737+0000","last_change":"2024-09-16T02:03:34.747828+0000","last_active":"2024-09-16T02:05:30.545737+0000","last_peered":"2024-09-16T02:05:30.545737+0000","last_clean":"2024-09-16T02:05:30.545737+0000","last_became_active":"2024-09-16T02:03:34.747548+0000","last_became_peered":"2024-09-16T02:03:34.747548+0000","last_unstale":"2024-09-16T02:05:30.545737+0000","last_undegraded":"2024-09-16T02:05:30.545737+0000","last_fullsized":"2024-09-16T02:05:30.545737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:49.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543830+0000","last_change":"2024-09-16T02:03:37.051477+0000","last_active":"2024-09-16T02:05:30.543830+0000","last_peered":"2024-09-16T02:05:30.543830+0000","last_clean":"2024-09-16T02:05:30.543830+0000","last_became_active":"2024-09-16T02:03:37.048063+0000","last_became_peered":"2024-09-16T02:03:37.048063+0000","last_unstale":"2024-09-16T02:05:30.543830+0000","last_undegraded":"2024-09-16T02:05:30.543830+0000","last_fullsized":"2024-09-16T02:05:30.543830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:15.973972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'14","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:31.053974+0000","last_change":"2024-09-16T02:03:34.742935+0000","last_active":"2024-09-16T02:05:31.053974+0000","last_peered":"2024-09-16T02:05:31.053974+0000","last_clean":"2024-09-16T02:05:31.053974+0000","last_became_active":"2024-09-16T02:03:34.742611+0000","last_became_peered":"2024-09-16T02:03:34.742611+0000","last_unstale":"2024-09-16T02:05:31.053974+0000","last_undegraded":"2024-09-16T02:05:31.053974+0000","last_fullsized":"2024-09-16T02:05:31.053974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:36.695220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926955+0000","last_change":"2024-09-16T02:03:37.054926+0000","last_active":"2024-09-16T02:05:30.926955+0000","last_peered":"2024-09-16T02:05:30.926955+0000","last_clean":"2024-09-16T02:05:30.926955+0000","last_became_active":"2024-09-16T02:03:37.054483+0000","last_became_peered":"2024-09-16T02:03:37.054483+0000","last_unstale":"2024-09-16T02:05:30.926955+0000","last_undegraded":"2024-09-16T02:05:30.926955+0000","last_fullsized":"2024-09-16T02:05:30.926955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545961+0000","last_change":"2024-09-16T02:03:34.739611+0000","last_active":"2024-09-16T02:05:30.545961+0000","last_peered":"2024-09-16T02:05:30.545961+0000","last_clean":"2024-09-16T02:05:30.545961+0000","last_became_active":"2024-09-16T02:03:34.738090+0000","last_became_peered":"2024-09-16T02:03:34.738090+0000","last_unstale":"2024-09-16T02:05:30.545961+0000","last_undegraded":"2024-09-16T02:05:30.545961+0000","last_fullsized":"2024-09-16T02:05:30.545961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:47.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847671+0000","last_change":"2024-09-16T02:03:37.053020+0000","last_active":"2024-09-16T02:05:30.847671+0000","last_peered":"2024-09-16T02:05:30.847671+0000","last_clean":"2024-09-16T02:05:30.847671+0000","last_became_active":"2024-09-16T02:03:37.052365+0000","last_became_peered":"2024-09-16T02:03:37.052365+0000","last_unstale":"2024-09-16T02:05:30.847671+0000","last_undegraded":"2024-09-16T02:05:30.847671+0000","last_fullsized":"2024-09-16T02:05:30.847671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:54.965028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549356+0000","last_change":"2024-09-16T02:03:34.740726+0000","last_active":"2024-09-16T02:05:30.549356+0000","last_peered":"2024-09-16T02:05:30.549356+0000","last_clean":"2024-09-16T02:05:30.549356+0000","last_became_active":"2024-09-16T02:03:34.740402+0000","last_became_peered":"2024-09-16T02:03:34.740402+0000","last_unstale":"2024-09-16T02:05:30.549356+0000","last_undegraded":"2024-09-16T02:05:30.549356+0000","last_fullsized":"2024-09-16T02:05:30.549356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:05.761646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545210+0000","last_change":"2024-09-16T02:03:37.044862+0000","last_active":"2024-09-16T02:05:30.545210+0000","last_peered":"2024-09-16T02:05:30.545210+0000","last_clean":"2024-09-16T02:05:30.545210+0000","last_became_active":"2024-09-16T02:03:37.044406+0000","last_became_peered":"2024-09-16T02:03:37.044406+0000","last_unstale":"2024-09-16T02:05:30.545210+0000","last_undegraded":"2024-09-16T02:05:30.545210+0000","last_fullsized":"2024-09-16T02:05:30.545210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:03.326380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926544+0000","last_change":"2024-09-16T02:03:34.745505+0000","last_active":"2024-09-16T02:05:30.926544+0000","last_peered":"2024-09-16T02:05:30.926544+0000","last_clean":"2024-09-16T02:05:30.926544+0000","last_became_active":"2024-09-16T02:03:34.745151+0000","last_became_peered":"2024-09-16T02:03:34.745151+0000","last_unstale":"2024-09-16T02:05:30.926544+0000","last_undegraded":"2024-09-16T02:05:30.926544+0000","last_fullsized":"2024-09-16T02:05:30.926544+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:57.401176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547672+0000","last_change":"2024-09-16T02:03:37.046692+0000","last_active":"2024-09-16T02:05:30.547672+0000","last_peered":"2024-09-16T02:05:30.547672+0000","last_clean":"2024-09-16T02:05:30.547672+0000","last_became_active":"2024-09-16T02:03:37.046302+0000","last_became_peered":"2024-09-16T02:03:37.046302+0000","last_unstale":"2024-09-16T02:05:30.547672+0000","last_undegraded":"2024-09-16T02:05:30.547672+0000","last_fullsized":"2024-09-16T02:05:30.547672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:47.727431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544513+0000","last_change":"2024-09-16T02:03:34.739847+0000","last_active":"2024-09-16T02:05:30.544513+0000","last_peered":"2024-09-16T02:05:30.544513+0000","last_clean":"2024-09-16T02:05:30.544513+0000","last_became_active":"2024-09-16T02:03:34.739514+0000","last_became_peered":"2024-09-16T02:03:34.739514+0000","last_unstale":"2024-09-16T02:05:30.544513+0000","last_undegraded":"2024-09-16T02:05:30.544513+0000","last_fullsized":"2024-09-16T02:05:30.544513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:44.617127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926659+0000","last_change":"2024-09-16T02:03:37.054910+0000","last_active":"2024-09-16T02:05:30.926659+0000","last_peered":"2024-09-16T02:05:30.926659+0000","last_clean":"2024-09-16T02:05:30.926659+0000","last_became_active":"2024-09-16T02:03:37.053852+0000","last_became_peered":"2024-09-16T02:03:37.053852+0000","last_unstale":"2024-09-16T02:05:30.926659+0000","last_undegraded":"2024-09-16T02:05:30.926659+0000","last_fullsized":"2024-09-16T02:05:30.926659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:29.042154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545928+0000","last_change":"2024-09-16T02:03:34.743027+0000","last_active":"2024-09-16T02:05:30.545928+0000","last_peered":"2024-09-16T02:05:30.545928+0000","last_clean":"2024-09-16T02:05:30.545928+0000","last_became_active":"2024-09-16T02:03:34.742658+0000","last_became_peered":"2024-09-16T02:03:34.742658+0000","last_unstale":"2024-09-16T02:05:30.545928+0000","last_undegraded":"2024-09-16T02:05:30.545928+0000","last_fullsized":"2024-09-16T02:05:30.545928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:36.779568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659557+0000","last_change":"2024-09-16T02:03:37.052783+0000","last_active":"2024-09-16T02:05:30.659557+0000","last_peered":"2024-09-16T02:05:30.659557+0000","last_clean":"2024-09-16T02:05:30.659557+0000","last_became_active":"2024-09-16T02:03:37.052409+0000","last_became_peered":"2024-09-16T02:03:37.052409+0000","last_unstale":"2024-09-16T02:05:30.659557+0000","last_undegraded":"2024-09-16T02:05:30.659557+0000","last_fullsized":"2024-09-16T02:05:30.659557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:52.667379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.551560+0000","last_change":"2024-09-16T02:05:30.551560+0000","last_active":"2024-09-16T02:05:30.551560+0000","last_peered":"2024-09-16T02:05:30.551560+0000","last_clean":"2024-09-16T02:05:30.551560+0000","last_became_active":"2024-09-16T02:05:30.551063+0000","last_became_peered":"2024-09-16T02:05:30.551063+0000","last_unstale":"2024-09-16T02:05:30.551560+0000","last_undegraded":"2024-09-16T02:05:30.551560+0000","last_fullsized":"2024-09-16T02:05:30.551560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:39.081149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545662+0000","last_change":"2024-09-16T02:03:34.744240+0000","last_active":"2024-09-16T02:05:30.545662+0000","last_peered":"2024-09-16T02:05:30.545662+0000","last_clean":"2024-09-16T02:05:30.545662+0000","last_became_active":"2024-09-16T02:03:34.743898+0000","last_became_peered":"2024-09-16T02:03:34.743898+0000","last_unstale":"2024-09-16T02:05:30.545662+0000","last_undegraded":"2024-09-16T02:05:30.545662+0000","last_fullsized":"2024-09-16T02:05:30.545662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:37.820529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549458+0000","last_change":"2024-09-16T02:03:37.055769+0000","last_active":"2024-09-16T02:05:30.549458+0000","last_peered":"2024-09-16T02:05:30.549458+0000","last_clean":"2024-09-16T02:05:30.549458+0000","last_became_active":"2024-09-16T02:03:37.054458+0000","last_became_peered":"2024-09-16T02:03:37.054458+0000","last_unstale":"2024-09-16T02:05:30.549458+0000","last_undegraded":"2024-09-16T02:05:30.549458+0000","last_fullsized":"2024-09-16T02:05:30.549458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:30.218712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545637+0000","last_change":"2024-09-16T02:04:29.647534+0000","last_active":"2024-09-16T02:05:30.545637+0000","last_peered":"2024-09-16T02:05:30.545637+0000","last_clean":"2024-09-16T02:05:30.545637+0000","last_became_active":"2024-09-16T02:04:29.640317+0000","last_became_peered":"2024-09-16T02:04:29.640317+0000","last_unstale":"2024-09-16T02:05:30.545637+0000","last_undegraded":"2024-09-16T02:05:30.545637+0000","last_fullsized":"2024-09-16T02:05:30.545637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:14:33.490508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,6],"acting":[8,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847438+0000","last_change":"2024-09-16T02:03:37.054370+0000","last_active":"2024-09-16T02:05:30.847438+0000","last_peered":"2024-09-16T02:05:30.847438+0000","last_clean":"2024-09-16T02:05:30.847438+0000","last_became_active":"2024-09-16T02:03:37.054051+0000","last_became_peered":"2024-09-16T02:03:37.054051+0000","last_unstale":"2024-09-16T02:05:30.847438+0000","last_undegraded":"2024-09-16T02:05:30.847438+0000","last_fullsized":"2024-09-16T02:05:30.847438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:21:42.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.927087+0000","last_change":"2024-09-16T02:03:34.739696+0000","last_active":"2024-09-16T02:05:30.927087+0000","last_peered":"2024-09-16T02:05:30.927087+0000","last_clean":"2024-09-16T02:05:30.927087+0000","last_became_active":"2024-09-16T02:03:34.738867+0000","last_became_peered":"2024-09-16T02:03:34.738867+0000","last_unstale":"2024-09-16T02:05:30.927087+0000","last_undegraded":"2024-09-16T02:05:30.927087+0000","last_fullsized":"2024-09-16T02:05:30.927087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:30.753970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926979+0000","last_change":"2024-09-16T02:03:37.058118+0000","last_active":"2024-09-16T02:05:30.926979+0000","last_peered":"2024-09-16T02:05:30.926979+0000","last_clean":"2024-09-16T02:05:30.926979+0000","last_became_active":"2024-09-16T02:03:37.057391+0000","last_became_peered":"2024-09-16T02:03:37.057391+0000","last_unstale":"2024-09-16T02:05:30.926979+0000","last_undegraded":"2024-09-16T02:05:30.926979+0000","last_fullsized":"2024-09-16T02:05:30.926979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:39.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548774+0000","last_change":"2024-09-16T02:03:34.742184+0000","last_active":"2024-09-16T02:05:30.548774+0000","last_peered":"2024-09-16T02:05:30.548774+0000","last_clean":"2024-09-16T02:05:30.548774+0000","last_became_active":"2024-09-16T02:03:34.741838+0000","last_became_peered":"2024-09-16T02:03:34.741838+0000","last_unstale":"2024-09-16T02:05:30.548774+0000","last_undegraded":"2024-09-16T02:05:30.548774+0000","last_fullsized":"2024-09-16T02:05:30.548774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:57.338380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65313,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":659,"num_read_kb":404,"num_write":181,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1149,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":207864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678054,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":440447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906985,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1597440,"data_stored":1092985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037807,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":440571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266742,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":440379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495674,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712488,"statfs":{"total":95995035648,"available":95961587712,"internally_reserved":0,"allocated":1511424,"data_stored":1033802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659199,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":905216,"data_stored":439960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888131,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1519616,"data_stored":1032226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051657,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":442398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280591,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":921600,"data_stored":440271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476820,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":506571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":970752,"data_stored":500099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":925696,"data_stored":441231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:05:36.830 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T02:05:37.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.019+0000 7ffa15e00640 1 -- 172.21.15.103:0/1536366622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa10076230 msgr2=0x7ffa10074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:37.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.019+0000 7ffa15e00640 1 --2- 172.21.15.103:0/1536366622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa10076230 0x7ffa10074680 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009900 tx=0x7ffa0002a410 comp rx=0 tx=0).stop 2024-09-16T02:05:37.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.019+0000 7ffa15e00640 1 -- 172.21.15.103:0/1536366622 shutdown_connections 2024-09-16T02:05:37.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.019+0000 7ffa15e00640 1 --2- 172.21.15.103:0/1536366622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa1010e0e0 0x7ffa101104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.019+0000 7ffa15e00640 1 --2- 172.21.15.103:0/1536366622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa10074c50 0x7ffa100750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.019+0000 7ffa15e00640 1 --2- 172.21.15.103:0/1536366622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa10076230 0x7ffa10074680 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.019+0000 7ffa15e00640 1 -- 172.21.15.103:0/1536366622 >> 172.21.15.103:0/1536366622 conn(0x7ffa100fd2b0 msgr2=0x7ffa100ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:37.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.019+0000 7ffa15e00640 1 -- 172.21.15.103:0/1536366622 shutdown_connections 2024-09-16T02:05:37.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.019+0000 7ffa15e00640 1 -- 172.21.15.103:0/1536366622 wait complete. 2024-09-16T02:05:37.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:36 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/4200589436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:37.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.020+0000 7ffa15e00640 1 Processor -- start 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.020+0000 7ffa15e00640 1 -- start start 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.020+0000 7ffa15e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa10074c50 0x7ffa1019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa15e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa10076230 0x7ffa1019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa15e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1010e0e0 0x7ffa101a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa15e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10113e30 con 0x7ffa1010e0e0 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa15e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10113cb0 con 0x7ffa10074c50 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa15e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10113fb0 con 0x7ffa10076230 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1010e0e0 0x7ffa101a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1010e0e0 0x7ffa101a59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54608/0 (socket says 172.21.15.103:54608) 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0f400640 1 -- 172.21.15.103:0/3539006267 learned_addr learned my addr 172.21.15.103:0/3539006267 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0ea00640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa10074c50 0x7ffa1019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:37.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0e000640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa10076230 0x7ffa1019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0f400640 1 -- 172.21.15.103:0/3539006267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa10076230 msgr2=0x7ffa1019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0f400640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa10076230 0x7ffa1019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0f400640 1 -- 172.21.15.103:0/3539006267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa10074c50 msgr2=0x7ffa1019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0f400640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa10074c50 0x7ffa1019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0f400640 1 -- 172.21.15.103:0/3539006267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa101a6020 con 0x7ffa1010e0e0 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.021+0000 7ffa0f400640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1010e0e0 0x7ffa101a59b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b970 tx=0x7ffa0800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.022+0000 7ffa14a00640 1 -- 172.21.15.103:0/3539006267 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08019070 con 0x7ffa1010e0e0 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.022+0000 7ffa14a00640 1 -- 172.21.15.103:0/3539006267 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa0800cb40 con 0x7ffa1010e0e0 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.022+0000 7ffa15e00640 1 -- 172.21.15.103:0/3539006267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa101a6310 con 0x7ffa1010e0e0 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.022+0000 7ffa14a00640 1 -- 172.21.15.103:0/3539006267 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08007650 con 0x7ffa1010e0e0 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.022+0000 7ffa15e00640 1 -- 172.21.15.103:0/3539006267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa101a6930 con 0x7ffa1010e0e0 2024-09-16T02:05:37.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.023+0000 7ffa15e00640 1 -- 172.21.15.103:0/3539006267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa10104190 con 0x7ffa1010e0e0 2024-09-16T02:05:37.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.025+0000 7ffa14a00640 1 -- 172.21.15.103:0/3539006267 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7ffa0801c070 con 0x7ffa1010e0e0 2024-09-16T02:05:37.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.026+0000 7ffa14a00640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f40a3360 0x7ff9f40a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:37.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.026+0000 7ffa14a00640 1 -- 172.21.15.103:0/3539006267 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7ffa080cef40 con 0x7ffa1010e0e0 2024-09-16T02:05:37.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.026+0000 7ffa0ea00640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f40a3360 0x7ff9f40a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:37.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.027+0000 7ffa0ea00640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f40a3360 0x7ff9f40a5820 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa00031440 tx=0x7ffa000315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:37.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.031+0000 7ffa14a00640 1 -- 172.21.15.103:0/3539006267 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa0809b540 con 0x7ffa1010e0e0 2024-09-16T02:05:37.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:36 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/4200589436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:37.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.206+0000 7ffa15e00640 1 -- 172.21.15.103:0/3539006267 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa10062630 con 0x7ff9f40a3360 2024-09-16T02:05:37.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.213+0000 7ffa14a00640 1 -- 172.21.15.103:0/3539006267 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258899 (secure 0 0 0) 0x7ffa10062630 con 0x7ff9f40a3360 2024-09-16T02:05:37.218 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:37.219 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-16T02:05:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 -- 172.21.15.103:0/3539006267 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f40a3360 msgr2=0x7ff9f40a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f40a3360 0x7ff9f40a5820 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa00031440 tx=0x7ffa000315b0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 -- 172.21.15.103:0/3539006267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1010e0e0 msgr2=0x7ffa101a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1010e0e0 0x7ffa101a59b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b970 tx=0x7ffa0800be30 comp rx=0 tx=0).stop 2024-09-16T02:05:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 -- 172.21.15.103:0/3539006267 shutdown_connections 2024-09-16T02:05:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff9f40a3360 0x7ff9f40a5820 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1010e0e0 0x7ffa101a59b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa10076230 0x7ffa1019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 --2- 172.21.15.103:0/3539006267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa10074c50 0x7ffa1019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 -- 172.21.15.103:0/3539006267 >> 172.21.15.103:0/3539006267 conn(0x7ffa100fd2b0 msgr2=0x7ffa10101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:37.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 -- 172.21.15.103:0/3539006267 shutdown_connections 2024-09-16T02:05:37.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.217+0000 7ffa15e00640 1 -- 172.21.15.103:0/3539006267 wait complete. 2024-09-16T02:05:37.235 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-16T02:05:36.478557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655915,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":785,"num_read_kb":513,"num_write":364,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181885,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":791,"ondisk_log_size":791,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370328,"kb_used_data":12688,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571496,"statfs":{"total":1151940427776,"available":1151561211904,"internally_reserved":0,"allocated":12992512,"data_stored":7250940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1153,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":25,"num_write":17,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002242"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547879+0000","last_change":"2024-09-16T02:03:37.050873+0000","last_active":"2024-09-16T02:05:30.547879+0000","last_peered":"2024-09-16T02:05:30.547879+0000","last_clean":"2024-09-16T02:05:30.547879+0000","last_became_active":"2024-09-16T02:03:37.050202+0000","last_became_peered":"2024-09-16T02:03:37.050202+0000","last_unstale":"2024-09-16T02:05:30.547879+0000","last_undegraded":"2024-09-16T02:05:30.547879+0000","last_fullsized":"2024-09-16T02:05:30.547879+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:12.971199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847625+0000","last_change":"2024-09-16T02:03:37.048034+0000","last_active":"2024-09-16T02:05:30.847625+0000","last_peered":"2024-09-16T02:05:30.847625+0000","last_clean":"2024-09-16T02:05:30.847625+0000","last_became_active":"2024-09-16T02:03:37.047216+0000","last_became_peered":"2024-09-16T02:03:37.047216+0000","last_unstale":"2024-09-16T02:05:30.847625+0000","last_undegraded":"2024-09-16T02:05:30.847625+0000","last_fullsized":"2024-09-16T02:05:30.847625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:48.580466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847483+0000","last_change":"2024-09-16T02:03:37.059287+0000","last_active":"2024-09-16T02:05:30.847483+0000","last_peered":"2024-09-16T02:05:30.847483+0000","last_clean":"2024-09-16T02:05:30.847483+0000","last_became_active":"2024-09-16T02:03:37.055270+0000","last_became_peered":"2024-09-16T02:03:37.055270+0000","last_unstale":"2024-09-16T02:05:30.847483+0000","last_undegraded":"2024-09-16T02:05:30.847483+0000","last_fullsized":"2024-09-16T02:05:30.847483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:29.430514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545655+0000","last_change":"2024-09-16T02:03:37.052473+0000","last_active":"2024-09-16T02:05:30.545655+0000","last_peered":"2024-09-16T02:05:30.545655+0000","last_clean":"2024-09-16T02:05:30.545655+0000","last_became_active":"2024-09-16T02:03:37.052015+0000","last_became_peered":"2024-09-16T02:03:37.052015+0000","last_unstale":"2024-09-16T02:05:30.545655+0000","last_undegraded":"2024-09-16T02:05:30.545655+0000","last_fullsized":"2024-09-16T02:05:30.545655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:03.964132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550684+0000","last_change":"2024-09-16T02:05:30.550684+0000","last_active":"2024-09-16T02:05:30.550684+0000","last_peered":"2024-09-16T02:05:30.550684+0000","last_clean":"2024-09-16T02:05:30.550684+0000","last_became_active":"2024-09-16T02:05:30.550245+0000","last_became_peered":"2024-09-16T02:05:30.550245+0000","last_unstale":"2024-09-16T02:05:30.550684+0000","last_undegraded":"2024-09-16T02:05:30.550684+0000","last_fullsized":"2024-09-16T02:05:30.550684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:06.367612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545873+0000","last_change":"2024-09-16T02:03:37.048912+0000","last_active":"2024-09-16T02:05:30.545873+0000","last_peered":"2024-09-16T02:05:30.545873+0000","last_clean":"2024-09-16T02:05:30.545873+0000","last_became_active":"2024-09-16T02:03:37.048577+0000","last_became_peered":"2024-09-16T02:03:37.048577+0000","last_unstale":"2024-09-16T02:05:30.545873+0000","last_undegraded":"2024-09-16T02:05:30.545873+0000","last_fullsized":"2024-09-16T02:05:30.545873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:25.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659580+0000","last_change":"2024-09-16T02:03:37.057427+0000","last_active":"2024-09-16T02:05:30.659580+0000","last_peered":"2024-09-16T02:05:30.659580+0000","last_clean":"2024-09-16T02:05:30.659580+0000","last_became_active":"2024-09-16T02:03:37.057111+0000","last_became_peered":"2024-09-16T02:03:37.057111+0000","last_unstale":"2024-09-16T02:05:30.659580+0000","last_undegraded":"2024-09-16T02:05:30.659580+0000","last_fullsized":"2024-09-16T02:05:30.659580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:44.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926373+0000","last_change":"2024-09-16T02:03:37.051085+0000","last_active":"2024-09-16T02:05:30.926373+0000","last_peered":"2024-09-16T02:05:30.926373+0000","last_clean":"2024-09-16T02:05:30.926373+0000","last_became_active":"2024-09-16T02:03:37.050790+0000","last_became_peered":"2024-09-16T02:03:37.050790+0000","last_unstale":"2024-09-16T02:05:30.926373+0000","last_undegraded":"2024-09-16T02:05:30.926373+0000","last_fullsized":"2024-09-16T02:05:30.926373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:18.620901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549870+0000","last_change":"2024-09-16T02:03:37.054258+0000","last_active":"2024-09-16T02:05:30.549870+0000","last_peered":"2024-09-16T02:05:30.549870+0000","last_clean":"2024-09-16T02:05:30.549870+0000","last_became_active":"2024-09-16T02:03:37.053939+0000","last_became_peered":"2024-09-16T02:03:37.053939+0000","last_unstale":"2024-09-16T02:05:30.549870+0000","last_undegraded":"2024-09-16T02:05:30.549870+0000","last_fullsized":"2024-09-16T02:05:30.549870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:11.076147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.546724+0000","last_change":"2024-09-16T02:03:37.049828+0000","last_active":"2024-09-16T02:05:30.546724+0000","last_peered":"2024-09-16T02:05:30.546724+0000","last_clean":"2024-09-16T02:05:30.546724+0000","last_became_active":"2024-09-16T02:03:37.049577+0000","last_became_peered":"2024-09-16T02:03:37.049577+0000","last_unstale":"2024-09-16T02:05:30.546724+0000","last_undegraded":"2024-09-16T02:05:30.546724+0000","last_fullsized":"2024-09-16T02:05:30.546724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:37.176431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659716+0000","last_change":"2024-09-16T02:03:37.053268+0000","last_active":"2024-09-16T02:05:30.659716+0000","last_peered":"2024-09-16T02:05:30.659716+0000","last_clean":"2024-09-16T02:05:30.659716+0000","last_became_active":"2024-09-16T02:03:37.052933+0000","last_became_peered":"2024-09-16T02:03:37.052933+0000","last_unstale":"2024-09-16T02:05:30.659716+0000","last_undegraded":"2024-09-16T02:05:30.659716+0000","last_fullsized":"2024-09-16T02:05:30.659716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:32.003728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545762+0000","last_change":"2024-09-16T02:03:37.046339+0000","last_active":"2024-09-16T02:05:30.545762+0000","last_peered":"2024-09-16T02:05:30.545762+0000","last_clean":"2024-09-16T02:05:30.545762+0000","last_became_active":"2024-09-16T02:03:37.043262+0000","last_became_peered":"2024-09-16T02:03:37.043262+0000","last_unstale":"2024-09-16T02:05:30.545762+0000","last_undegraded":"2024-09-16T02:05:30.545762+0000","last_fullsized":"2024-09-16T02:05:30.545762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:58.614672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550406+0000","last_change":"2024-09-16T02:05:30.550406+0000","last_active":"2024-09-16T02:05:30.550406+0000","last_peered":"2024-09-16T02:05:30.550406+0000","last_clean":"2024-09-16T02:05:30.550406+0000","last_became_active":"2024-09-16T02:05:30.549945+0000","last_became_peered":"2024-09-16T02:05:30.549945+0000","last_unstale":"2024-09-16T02:05:30.550406+0000","last_undegraded":"2024-09-16T02:05:30.550406+0000","last_fullsized":"2024-09-16T02:05:30.550406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:49.789795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847255+0000","last_change":"2024-09-16T02:03:37.048844+0000","last_active":"2024-09-16T02:05:30.847255+0000","last_peered":"2024-09-16T02:05:30.847255+0000","last_clean":"2024-09-16T02:05:30.847255+0000","last_became_active":"2024-09-16T02:03:37.048522+0000","last_became_peered":"2024-09-16T02:03:37.048522+0000","last_unstale":"2024-09-16T02:05:30.847255+0000","last_undegraded":"2024-09-16T02:05:30.847255+0000","last_fullsized":"2024-09-16T02:05:30.847255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:50.759847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847176+0000","last_change":"2024-09-16T02:03:37.043236+0000","last_active":"2024-09-16T02:05:30.847176+0000","last_peered":"2024-09-16T02:05:30.847176+0000","last_clean":"2024-09-16T02:05:30.847176+0000","last_became_active":"2024-09-16T02:03:37.042967+0000","last_became_peered":"2024-09-16T02:03:37.042967+0000","last_unstale":"2024-09-16T02:05:30.847176+0000","last_undegraded":"2024-09-16T02:05:30.847176+0000","last_fullsized":"2024-09-16T02:05:30.847176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:02.739678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549252+0000","last_change":"2024-09-16T02:03:37.052972+0000","last_active":"2024-09-16T02:05:30.549252+0000","last_peered":"2024-09-16T02:05:30.549252+0000","last_clean":"2024-09-16T02:05:30.549252+0000","last_became_active":"2024-09-16T02:03:37.052665+0000","last_became_peered":"2024-09-16T02:03:37.052665+0000","last_unstale":"2024-09-16T02:05:30.549252+0000","last_undegraded":"2024-09-16T02:05:30.549252+0000","last_fullsized":"2024-09-16T02:05:30.549252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:59.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547979+0000","last_change":"2024-09-16T02:03:37.051203+0000","last_active":"2024-09-16T02:05:30.547979+0000","last_peered":"2024-09-16T02:05:30.547979+0000","last_clean":"2024-09-16T02:05:30.547979+0000","last_became_active":"2024-09-16T02:03:37.049828+0000","last_became_peered":"2024-09-16T02:03:37.049828+0000","last_unstale":"2024-09-16T02:05:30.547979+0000","last_undegraded":"2024-09-16T02:05:30.547979+0000","last_fullsized":"2024-09-16T02:05:30.547979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:47.566594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548575+0000","last_change":"2024-09-16T02:03:37.056903+0000","last_active":"2024-09-16T02:05:30.548575+0000","last_peered":"2024-09-16T02:05:30.548575+0000","last_clean":"2024-09-16T02:05:30.548575+0000","last_became_active":"2024-09-16T02:03:37.056579+0000","last_became_peered":"2024-09-16T02:03:37.056579+0000","last_unstale":"2024-09-16T02:05:30.548575+0000","last_undegraded":"2024-09-16T02:05:30.548575+0000","last_fullsized":"2024-09-16T02:05:30.548575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:23.652294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544817+0000","last_change":"2024-09-16T02:03:37.054413+0000","last_active":"2024-09-16T02:05:30.544817+0000","last_peered":"2024-09-16T02:05:30.544817+0000","last_clean":"2024-09-16T02:05:30.544817+0000","last_became_active":"2024-09-16T02:03:37.054040+0000","last_became_peered":"2024-09-16T02:03:37.054040+0000","last_unstale":"2024-09-16T02:05:30.544817+0000","last_undegraded":"2024-09-16T02:05:30.544817+0000","last_fullsized":"2024-09-16T02:05:30.544817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:04.517313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547502+0000","last_change":"2024-09-16T02:03:37.053454+0000","last_active":"2024-09-16T02:05:30.547502+0000","last_peered":"2024-09-16T02:05:30.547502+0000","last_clean":"2024-09-16T02:05:30.547502+0000","last_became_active":"2024-09-16T02:03:37.053123+0000","last_became_peered":"2024-09-16T02:03:37.053123+0000","last_unstale":"2024-09-16T02:05:30.547502+0000","last_undegraded":"2024-09-16T02:05:30.547502+0000","last_fullsized":"2024-09-16T02:05:30.547502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:52.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543738+0000","last_change":"2024-09-16T02:03:37.042772+0000","last_active":"2024-09-16T02:05:30.543738+0000","last_peered":"2024-09-16T02:05:30.543738+0000","last_clean":"2024-09-16T02:05:30.543738+0000","last_became_active":"2024-09-16T02:03:37.041832+0000","last_became_peered":"2024-09-16T02:03:37.041832+0000","last_unstale":"2024-09-16T02:05:30.543738+0000","last_undegraded":"2024-09-16T02:05:30.543738+0000","last_fullsized":"2024-09-16T02:05:30.543738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:54.436291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549491+0000","last_change":"2024-09-16T02:03:37.058484+0000","last_active":"2024-09-16T02:05:30.549491+0000","last_peered":"2024-09-16T02:05:30.549491+0000","last_clean":"2024-09-16T02:05:30.549491+0000","last_became_active":"2024-09-16T02:03:37.058047+0000","last_became_peered":"2024-09-16T02:03:37.058047+0000","last_unstale":"2024-09-16T02:05:30.549491+0000","last_undegraded":"2024-09-16T02:05:30.549491+0000","last_fullsized":"2024-09-16T02:05:30.549491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:23.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926399+0000","last_change":"2024-09-16T02:03:37.058576+0000","last_active":"2024-09-16T02:05:30.926399+0000","last_peered":"2024-09-16T02:05:30.926399+0000","last_clean":"2024-09-16T02:05:30.926399+0000","last_became_active":"2024-09-16T02:03:37.057624+0000","last_became_peered":"2024-09-16T02:03:37.057624+0000","last_unstale":"2024-09-16T02:05:30.926399+0000","last_undegraded":"2024-09-16T02:05:30.926399+0000","last_fullsized":"2024-09-16T02:05:30.926399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:21.929461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549455+0000","last_change":"2024-09-16T02:03:37.050889+0000","last_active":"2024-09-16T02:05:30.549455+0000","last_peered":"2024-09-16T02:05:30.549455+0000","last_clean":"2024-09-16T02:05:30.549455+0000","last_became_active":"2024-09-16T02:03:37.050535+0000","last_became_peered":"2024-09-16T02:03:37.050535+0000","last_unstale":"2024-09-16T02:05:30.549455+0000","last_undegraded":"2024-09-16T02:05:30.549455+0000","last_fullsized":"2024-09-16T02:05:30.549455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:46.955962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926844+0000","last_change":"2024-09-16T02:03:37.053865+0000","last_active":"2024-09-16T02:05:30.926844+0000","last_peered":"2024-09-16T02:05:30.926844+0000","last_clean":"2024-09-16T02:05:30.926844+0000","last_became_active":"2024-09-16T02:03:37.053563+0000","last_became_peered":"2024-09-16T02:03:37.053563+0000","last_unstale":"2024-09-16T02:05:30.926844+0000","last_undegraded":"2024-09-16T02:05:30.926844+0000","last_fullsized":"2024-09-16T02:05:30.926844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:24.109471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545076+0000","last_change":"2024-09-16T02:03:37.054681+0000","last_active":"2024-09-16T02:05:30.545076+0000","last_peered":"2024-09-16T02:05:30.545076+0000","last_clean":"2024-09-16T02:05:30.545076+0000","last_became_active":"2024-09-16T02:03:37.054324+0000","last_became_peered":"2024-09-16T02:03:37.054324+0000","last_unstale":"2024-09-16T02:05:30.545076+0000","last_undegraded":"2024-09-16T02:05:30.545076+0000","last_fullsized":"2024-09-16T02:05:30.545076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:20.472520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547733+0000","last_change":"2024-09-16T02:03:37.047713+0000","last_active":"2024-09-16T02:05:30.547733+0000","last_peered":"2024-09-16T02:05:30.547733+0000","last_clean":"2024-09-16T02:05:30.547733+0000","last_became_active":"2024-09-16T02:03:37.047353+0000","last_became_peered":"2024-09-16T02:03:37.047353+0000","last_unstale":"2024-09-16T02:05:30.547733+0000","last_undegraded":"2024-09-16T02:05:30.547733+0000","last_fullsized":"2024-09-16T02:05:30.547733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:05.868909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543731+0000","last_change":"2024-09-16T02:03:37.038931+0000","last_active":"2024-09-16T02:05:30.543731+0000","last_peered":"2024-09-16T02:05:30.543731+0000","last_clean":"2024-09-16T02:05:30.543731+0000","last_became_active":"2024-09-16T02:03:37.037234+0000","last_became_peered":"2024-09-16T02:03:37.037234+0000","last_unstale":"2024-09-16T02:05:30.543731+0000","last_undegraded":"2024-09-16T02:05:30.543731+0000","last_fullsized":"2024-09-16T02:05:30.543731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:54.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847518+0000","last_change":"2024-09-16T02:04:29.634327+0000","last_active":"2024-09-16T02:05:30.847518+0000","last_peered":"2024-09-16T02:05:30.847518+0000","last_clean":"2024-09-16T02:05:30.847518+0000","last_became_active":"2024-09-16T02:04:29.633710+0000","last_became_peered":"2024-09-16T02:04:29.633710+0000","last_unstale":"2024-09-16T02:05:30.847518+0000","last_undegraded":"2024-09-16T02:05:30.847518+0000","last_fullsized":"2024-09-16T02:05:30.847518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:55:03.574092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544503+0000","last_change":"2024-09-16T02:03:37.057688+0000","last_active":"2024-09-16T02:05:30.544503+0000","last_peered":"2024-09-16T02:05:30.544503+0000","last_clean":"2024-09-16T02:05:30.544503+0000","last_became_active":"2024-09-16T02:03:37.057347+0000","last_became_peered":"2024-09-16T02:03:37.057347+0000","last_unstale":"2024-09-16T02:05:30.544503+0000","last_undegraded":"2024-09-16T02:05:30.544503+0000","last_fullsized":"2024-09-16T02:05:30.544503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:30.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544749+0000","last_change":"2024-09-16T02:04:29.640455+0000","last_active":"2024-09-16T02:05:30.544749+0000","last_peered":"2024-09-16T02:05:30.544749+0000","last_clean":"2024-09-16T02:05:30.544749+0000","last_became_active":"2024-09-16T02:04:29.640083+0000","last_became_peered":"2024-09-16T02:04:29.640083+0000","last_unstale":"2024-09-16T02:05:30.544749+0000","last_undegraded":"2024-09-16T02:05:30.544749+0000","last_fullsized":"2024-09-16T02:05:30.544749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:46.191484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547527+0000","last_change":"2024-09-16T02:03:37.043596+0000","last_active":"2024-09-16T02:05:30.547527+0000","last_peered":"2024-09-16T02:05:30.547527+0000","last_clean":"2024-09-16T02:05:30.547527+0000","last_became_active":"2024-09-16T02:03:37.043261+0000","last_became_peered":"2024-09-16T02:03:37.043261+0000","last_unstale":"2024-09-16T02:05:30.547527+0000","last_undegraded":"2024-09-16T02:05:30.547527+0000","last_fullsized":"2024-09-16T02:05:30.547527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:03.833126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545258+0000","last_change":"2024-09-16T02:03:37.043018+0000","last_active":"2024-09-16T02:05:30.545258+0000","last_peered":"2024-09-16T02:05:30.545258+0000","last_clean":"2024-09-16T02:05:30.545258+0000","last_became_active":"2024-09-16T02:03:37.042437+0000","last_became_peered":"2024-09-16T02:03:37.042437+0000","last_unstale":"2024-09-16T02:05:30.545258+0000","last_undegraded":"2024-09-16T02:05:30.545258+0000","last_fullsized":"2024-09-16T02:05:30.545258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.193852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'7","reported_seq":153,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.739530+0000","last_change":"2024-09-16T02:04:29.645481+0000","last_active":"2024-09-16T02:05:30.739530+0000","last_peered":"2024-09-16T02:05:30.739530+0000","last_clean":"2024-09-16T02:05:30.739530+0000","last_became_active":"2024-09-16T02:04:29.640089+0000","last_became_peered":"2024-09-16T02:04:29.640089+0000","last_unstale":"2024-09-16T02:05:30.739530+0000","last_undegraded":"2024-09-16T02:05:30.739530+0000","last_fullsized":"2024-09-16T02:05:30.739530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:02.431443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549918+0000","last_change":"2024-09-16T02:03:37.052664+0000","last_active":"2024-09-16T02:05:30.549918+0000","last_peered":"2024-09-16T02:05:30.549918+0000","last_clean":"2024-09-16T02:05:30.549918+0000","last_became_active":"2024-09-16T02:03:37.051310+0000","last_became_peered":"2024-09-16T02:03:37.051310+0000","last_unstale":"2024-09-16T02:05:30.549918+0000","last_undegraded":"2024-09-16T02:05:30.549918+0000","last_fullsized":"2024-09-16T02:05:30.549918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:57.979053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547503+0000","last_change":"2024-09-16T02:04:29.654377+0000","last_active":"2024-09-16T02:05:30.547503+0000","last_peered":"2024-09-16T02:05:30.547503+0000","last_clean":"2024-09-16T02:05:30.547503+0000","last_became_active":"2024-09-16T02:04:29.635187+0000","last_became_peered":"2024-09-16T02:04:29.635187+0000","last_unstale":"2024-09-16T02:05:30.547503+0000","last_undegraded":"2024-09-16T02:05:30.547503+0000","last_fullsized":"2024-09-16T02:05:30.547503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:28.474841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,6],"acting":[11,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543512+0000","last_change":"2024-09-16T02:03:34.745325+0000","last_active":"2024-09-16T02:05:30.543512+0000","last_peered":"2024-09-16T02:05:30.543512+0000","last_clean":"2024-09-16T02:05:30.543512+0000","last_became_active":"2024-09-16T02:03:34.744730+0000","last_became_peered":"2024-09-16T02:03:34.744730+0000","last_unstale":"2024-09-16T02:05:30.543512+0000","last_undegraded":"2024-09-16T02:05:30.543512+0000","last_fullsized":"2024-09-16T02:05:30.543512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926402+0000","last_change":"2024-09-16T02:03:37.058973+0000","last_active":"2024-09-16T02:05:30.926402+0000","last_peered":"2024-09-16T02:05:30.926402+0000","last_clean":"2024-09-16T02:05:30.926402+0000","last_became_active":"2024-09-16T02:03:37.058685+0000","last_became_peered":"2024-09-16T02:03:37.058685+0000","last_unstale":"2024-09-16T02:05:30.926402+0000","last_undegraded":"2024-09-16T02:05:30.926402+0000","last_fullsized":"2024-09-16T02:05:30.926402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:46.030440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549707+0000","last_change":"2024-09-16T02:03:34.737842+0000","last_active":"2024-09-16T02:05:30.549707+0000","last_peered":"2024-09-16T02:05:30.549707+0000","last_clean":"2024-09-16T02:05:30.549707+0000","last_became_active":"2024-09-16T02:03:34.737391+0000","last_became_peered":"2024-09-16T02:03:34.737391+0000","last_unstale":"2024-09-16T02:05:30.549707+0000","last_undegraded":"2024-09-16T02:05:30.549707+0000","last_fullsized":"2024-09-16T02:05:30.549707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:39.214180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548437+0000","last_change":"2024-09-16T02:03:37.049705+0000","last_active":"2024-09-16T02:05:30.548437+0000","last_peered":"2024-09-16T02:05:30.548437+0000","last_clean":"2024-09-16T02:05:30.548437+0000","last_became_active":"2024-09-16T02:03:37.049392+0000","last_became_peered":"2024-09-16T02:03:37.049392+0000","last_unstale":"2024-09-16T02:05:30.548437+0000","last_undegraded":"2024-09-16T02:05:30.548437+0000","last_fullsized":"2024-09-16T02:05:30.548437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.932391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:31.740755+0000","last_change":"2024-09-16T02:03:34.742268+0000","last_active":"2024-09-16T02:05:31.740755+0000","last_peered":"2024-09-16T02:05:31.740755+0000","last_clean":"2024-09-16T02:05:31.740755+0000","last_became_active":"2024-09-16T02:03:34.741934+0000","last_became_peered":"2024-09-16T02:03:34.741934+0000","last_unstale":"2024-09-16T02:05:31.740755+0000","last_undegraded":"2024-09-16T02:05:31.740755+0000","last_fullsized":"2024-09-16T02:05:31.740755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:30.301552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549786+0000","last_change":"2024-09-16T02:03:37.059119+0000","last_active":"2024-09-16T02:05:30.549786+0000","last_peered":"2024-09-16T02:05:30.549786+0000","last_clean":"2024-09-16T02:05:30.549786+0000","last_became_active":"2024-09-16T02:03:37.058738+0000","last_became_peered":"2024-09-16T02:03:37.058738+0000","last_unstale":"2024-09-16T02:05:30.549786+0000","last_undegraded":"2024-09-16T02:05:30.549786+0000","last_fullsized":"2024-09-16T02:05:30.549786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.496053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659981+0000","last_change":"2024-09-16T02:04:29.642379+0000","last_active":"2024-09-16T02:05:30.659981+0000","last_peered":"2024-09-16T02:05:30.659981+0000","last_clean":"2024-09-16T02:05:30.659981+0000","last_became_active":"2024-09-16T02:04:29.638587+0000","last_became_peered":"2024-09-16T02:04:29.638587+0000","last_unstale":"2024-09-16T02:05:30.659981+0000","last_undegraded":"2024-09-16T02:05:30.659981+0000","last_fullsized":"2024-09-16T02:05:30.659981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:50.268223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847445+0000","last_change":"2024-09-16T02:03:37.050349+0000","last_active":"2024-09-16T02:05:30.847445+0000","last_peered":"2024-09-16T02:05:30.847445+0000","last_clean":"2024-09-16T02:05:30.847445+0000","last_became_active":"2024-09-16T02:03:37.049978+0000","last_became_peered":"2024-09-16T02:03:37.049978+0000","last_unstale":"2024-09-16T02:05:30.847445+0000","last_undegraded":"2024-09-16T02:05:30.847445+0000","last_fullsized":"2024-09-16T02:05:30.847445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:45.119683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550509+0000","last_change":"2024-09-16T02:03:34.740246+0000","last_active":"2024-09-16T02:05:30.550509+0000","last_peered":"2024-09-16T02:05:30.550509+0000","last_clean":"2024-09-16T02:05:30.550509+0000","last_became_active":"2024-09-16T02:03:34.739908+0000","last_became_peered":"2024-09-16T02:03:34.739908+0000","last_unstale":"2024-09-16T02:05:30.550509+0000","last_undegraded":"2024-09-16T02:05:30.550509+0000","last_fullsized":"2024-09-16T02:05:30.550509+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:35.299733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549580+0000","last_change":"2024-09-16T02:03:37.056562+0000","last_active":"2024-09-16T02:05:30.549580+0000","last_peered":"2024-09-16T02:05:30.549580+0000","last_clean":"2024-09-16T02:05:30.549580+0000","last_became_active":"2024-09-16T02:03:37.056226+0000","last_became_peered":"2024-09-16T02:03:37.056226+0000","last_unstale":"2024-09-16T02:05:30.549580+0000","last_undegraded":"2024-09-16T02:05:30.549580+0000","last_fullsized":"2024-09-16T02:05:30.549580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.741538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":279,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:32.742423+0000","last_change":"2024-09-16T02:03:34.736403+0000","last_active":"2024-09-16T02:05:32.742423+0000","last_peered":"2024-09-16T02:05:32.742423+0000","last_clean":"2024-09-16T02:05:32.742423+0000","last_became_active":"2024-09-16T02:03:34.736074+0000","last_became_peered":"2024-09-16T02:03:34.736074+0000","last_unstale":"2024-09-16T02:05:32.742423+0000","last_undegraded":"2024-09-16T02:05:32.742423+0000","last_fullsized":"2024-09-16T02:05:32.742423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:34.769862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60324,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":169,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545616+0000","last_change":"2024-09-16T02:03:37.046803+0000","last_active":"2024-09-16T02:05:30.545616+0000","last_peered":"2024-09-16T02:05:30.545616+0000","last_clean":"2024-09-16T02:05:30.545616+0000","last_became_active":"2024-09-16T02:03:37.046459+0000","last_became_peered":"2024-09-16T02:03:37.046459+0000","last_unstale":"2024-09-16T02:05:30.545616+0000","last_undegraded":"2024-09-16T02:05:30.545616+0000","last_fullsized":"2024-09-16T02:05:30.545616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:55:38.460795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544547+0000","last_change":"2024-09-16T02:03:34.739016+0000","last_active":"2024-09-16T02:05:30.544547+0000","last_peered":"2024-09-16T02:05:30.544547+0000","last_clean":"2024-09-16T02:05:30.544547+0000","last_became_active":"2024-09-16T02:03:34.738583+0000","last_became_peered":"2024-09-16T02:03:34.738583+0000","last_unstale":"2024-09-16T02:05:30.544547+0000","last_undegraded":"2024-09-16T02:05:30.544547+0000","last_fullsized":"2024-09-16T02:05:30.544547+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:08.255832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543487+0000","last_change":"2024-09-16T02:03:37.053353+0000","last_active":"2024-09-16T02:05:30.543487+0000","last_peered":"2024-09-16T02:05:30.543487+0000","last_clean":"2024-09-16T02:05:30.543487+0000","last_became_active":"2024-09-16T02:03:37.051140+0000","last_became_peered":"2024-09-16T02:03:37.051140+0000","last_unstale":"2024-09-16T02:05:30.543487+0000","last_undegraded":"2024-09-16T02:05:30.543487+0000","last_fullsized":"2024-09-16T02:05:30.543487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:34.636338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549839+0000","last_change":"2024-09-16T02:03:34.742111+0000","last_active":"2024-09-16T02:05:30.549839+0000","last_peered":"2024-09-16T02:05:30.549839+0000","last_clean":"2024-09-16T02:05:30.549839+0000","last_became_active":"2024-09-16T02:03:34.741816+0000","last_became_peered":"2024-09-16T02:03:34.741816+0000","last_unstale":"2024-09-16T02:05:30.549839+0000","last_undegraded":"2024-09-16T02:05:30.549839+0000","last_fullsized":"2024-09-16T02:05:30.549839+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:36.861995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659571+0000","last_change":"2024-09-16T02:03:37.047152+0000","last_active":"2024-09-16T02:05:30.659571+0000","last_peered":"2024-09-16T02:05:30.659571+0000","last_clean":"2024-09-16T02:05:30.659571+0000","last_became_active":"2024-09-16T02:03:37.046675+0000","last_became_peered":"2024-09-16T02:03:37.046675+0000","last_unstale":"2024-09-16T02:05:30.659571+0000","last_undegraded":"2024-09-16T02:05:30.659571+0000","last_fullsized":"2024-09-16T02:05:30.659571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:58.236519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":142,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:31.267665+0000","last_change":"2024-09-16T02:03:34.738487+0000","last_active":"2024-09-16T02:05:31.267665+0000","last_peered":"2024-09-16T02:05:31.267665+0000","last_clean":"2024-09-16T02:05:31.267665+0000","last_became_active":"2024-09-16T02:03:34.737061+0000","last_became_peered":"2024-09-16T02:03:34.737061+0000","last_unstale":"2024-09-16T02:05:31.267665+0000","last_undegraded":"2024-09-16T02:05:31.267665+0000","last_fullsized":"2024-09-16T02:05:31.267665+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:24.231640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1433,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659353+0000","last_change":"2024-09-16T02:03:37.044260+0000","last_active":"2024-09-16T02:05:30.659353+0000","last_peered":"2024-09-16T02:05:30.659353+0000","last_clean":"2024-09-16T02:05:30.659353+0000","last_became_active":"2024-09-16T02:03:37.043932+0000","last_became_peered":"2024-09-16T02:03:37.043932+0000","last_unstale":"2024-09-16T02:05:30.659353+0000","last_undegraded":"2024-09-16T02:05:30.659353+0000","last_fullsized":"2024-09-16T02:05:30.659353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:55.718155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547422+0000","last_change":"2024-09-16T02:04:29.651235+0000","last_active":"2024-09-16T02:05:30.547422+0000","last_peered":"2024-09-16T02:05:30.547422+0000","last_clean":"2024-09-16T02:05:30.547422+0000","last_became_active":"2024-09-16T02:04:29.633999+0000","last_became_peered":"2024-09-16T02:04:29.633999+0000","last_unstale":"2024-09-16T02:05:30.547422+0000","last_undegraded":"2024-09-16T02:05:30.547422+0000","last_fullsized":"2024-09-16T02:05:30.547422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.616990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549533+0000","last_change":"2024-09-16T02:03:37.053040+0000","last_active":"2024-09-16T02:05:30.549533+0000","last_peered":"2024-09-16T02:05:30.549533+0000","last_clean":"2024-09-16T02:05:30.549533+0000","last_became_active":"2024-09-16T02:03:37.052185+0000","last_became_peered":"2024-09-16T02:03:37.052185+0000","last_unstale":"2024-09-16T02:05:30.549533+0000","last_undegraded":"2024-09-16T02:05:30.549533+0000","last_fullsized":"2024-09-16T02:05:30.549533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:27.676678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'3","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545839+0000","last_change":"2024-09-16T02:04:29.651376+0000","last_active":"2024-09-16T02:05:30.545839+0000","last_peered":"2024-09-16T02:05:30.545839+0000","last_clean":"2024-09-16T02:05:30.545839+0000","last_became_active":"2024-09-16T02:04:29.640260+0000","last_became_peered":"2024-09-16T02:04:29.640260+0000","last_unstale":"2024-09-16T02:05:30.545839+0000","last_undegraded":"2024-09-16T02:05:30.545839+0000","last_fullsized":"2024-09-16T02:05:30.545839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:39.426183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.661732+0000","last_change":"2024-09-16T02:05:30.661732+0000","last_active":"2024-09-16T02:05:30.661732+0000","last_peered":"2024-09-16T02:05:30.661732+0000","last_clean":"2024-09-16T02:05:30.661732+0000","last_became_active":"2024-09-16T02:05:30.661261+0000","last_became_peered":"2024-09-16T02:05:30.661261+0000","last_unstale":"2024-09-16T02:05:30.661732+0000","last_undegraded":"2024-09-16T02:05:30.661732+0000","last_fullsized":"2024-09-16T02:05:30.661732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:10.510530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543828+0000","last_change":"2024-09-16T02:03:34.738516+0000","last_active":"2024-09-16T02:05:30.543828+0000","last_peered":"2024-09-16T02:05:30.543828+0000","last_clean":"2024-09-16T02:05:30.543828+0000","last_became_active":"2024-09-16T02:03:34.737701+0000","last_became_peered":"2024-09-16T02:03:34.737701+0000","last_unstale":"2024-09-16T02:05:30.543828+0000","last_undegraded":"2024-09-16T02:05:30.543828+0000","last_fullsized":"2024-09-16T02:05:30.543828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:42.101865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659922+0000","last_change":"2024-09-16T02:03:37.046820+0000","last_active":"2024-09-16T02:05:30.659922+0000","last_peered":"2024-09-16T02:05:30.659922+0000","last_clean":"2024-09-16T02:05:30.659922+0000","last_became_active":"2024-09-16T02:03:37.046082+0000","last_became_peered":"2024-09-16T02:03:37.046082+0000","last_unstale":"2024-09-16T02:05:30.659922+0000","last_undegraded":"2024-09-16T02:05:30.659922+0000","last_fullsized":"2024-09-16T02:05:30.659922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:21.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550670+0000","last_change":"2024-09-16T02:03:34.741908+0000","last_active":"2024-09-16T02:05:30.550670+0000","last_peered":"2024-09-16T02:05:30.550670+0000","last_clean":"2024-09-16T02:05:30.550670+0000","last_became_active":"2024-09-16T02:03:34.739826+0000","last_became_peered":"2024-09-16T02:03:34.739826+0000","last_unstale":"2024-09-16T02:05:30.550670+0000","last_undegraded":"2024-09-16T02:05:30.550670+0000","last_fullsized":"2024-09-16T02:05:30.550670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:04.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659551+0000","last_change":"2024-09-16T02:03:37.056459+0000","last_active":"2024-09-16T02:05:30.659551+0000","last_peered":"2024-09-16T02:05:30.659551+0000","last_clean":"2024-09-16T02:05:30.659551+0000","last_became_active":"2024-09-16T02:03:37.055902+0000","last_became_peered":"2024-09-16T02:03:37.055902+0000","last_unstale":"2024-09-16T02:05:30.659551+0000","last_undegraded":"2024-09-16T02:05:30.659551+0000","last_fullsized":"2024-09-16T02:05:30.659551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:48:30.815654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544898+0000","last_change":"2024-09-16T02:03:34.741207+0000","last_active":"2024-09-16T02:05:30.544898+0000","last_peered":"2024-09-16T02:05:30.544898+0000","last_clean":"2024-09-16T02:05:30.544898+0000","last_became_active":"2024-09-16T02:03:34.740523+0000","last_became_peered":"2024-09-16T02:03:34.740523+0000","last_unstale":"2024-09-16T02:05:30.544898+0000","last_undegraded":"2024-09-16T02:05:30.544898+0000","last_fullsized":"2024-09-16T02:05:30.544898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:36.199332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659815+0000","last_change":"2024-09-16T02:03:37.052792+0000","last_active":"2024-09-16T02:05:30.659815+0000","last_peered":"2024-09-16T02:05:30.659815+0000","last_clean":"2024-09-16T02:05:30.659815+0000","last_became_active":"2024-09-16T02:03:37.052301+0000","last_became_peered":"2024-09-16T02:03:37.052301+0000","last_unstale":"2024-09-16T02:05:30.659815+0000","last_undegraded":"2024-09-16T02:05:30.659815+0000","last_fullsized":"2024-09-16T02:05:30.659815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:55.197481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":338,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659545+0000","last_change":"2024-09-16T02:02:21.095893+0000","last_active":"2024-09-16T02:05:30.659545+0000","last_peered":"2024-09-16T02:05:30.659545+0000","last_clean":"2024-09-16T02:05:30.659545+0000","last_became_active":"2024-09-16T02:02:21.095395+0000","last_became_peered":"2024-09-16T02:02:21.095395+0000","last_unstale":"2024-09-16T02:05:30.659545+0000","last_undegraded":"2024-09-16T02:05:30.659545+0000","last_fullsized":"2024-09-16T02:05:30.659545+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_clean_scrub_stamp":"2024-09-16T01:58:50.954254+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:40.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545471+0000","last_change":"2024-09-16T02:03:34.746154+0000","last_active":"2024-09-16T02:05:30.545471+0000","last_peered":"2024-09-16T02:05:30.545471+0000","last_clean":"2024-09-16T02:05:30.545471+0000","last_became_active":"2024-09-16T02:03:34.745646+0000","last_became_peered":"2024-09-16T02:03:34.745646+0000","last_unstale":"2024-09-16T02:05:30.545471+0000","last_undegraded":"2024-09-16T02:05:30.545471+0000","last_fullsized":"2024-09-16T02:05:30.545471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:06.790000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926706+0000","last_change":"2024-09-16T02:03:37.056638+0000","last_active":"2024-09-16T02:05:30.926706+0000","last_peered":"2024-09-16T02:05:30.926706+0000","last_clean":"2024-09-16T02:05:30.926706+0000","last_became_active":"2024-09-16T02:03:37.056043+0000","last_became_peered":"2024-09-16T02:03:37.056043+0000","last_unstale":"2024-09-16T02:05:30.926706+0000","last_undegraded":"2024-09-16T02:05:30.926706+0000","last_fullsized":"2024-09-16T02:05:30.926706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:35:48.353380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659548+0000","last_change":"2024-09-16T02:04:29.647682+0000","last_active":"2024-09-16T02:05:30.659548+0000","last_peered":"2024-09-16T02:05:30.659548+0000","last_clean":"2024-09-16T02:05:30.659548+0000","last_became_active":"2024-09-16T02:04:29.640065+0000","last_became_peered":"2024-09-16T02:04:29.640065+0000","last_unstale":"2024-09-16T02:05:30.659548+0000","last_undegraded":"2024-09-16T02:05:30.659548+0000","last_fullsized":"2024-09-16T02:05:30.659548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:53.765121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543418+0000","last_change":"2024-09-16T02:03:37.050895+0000","last_active":"2024-09-16T02:05:30.543418+0000","last_peered":"2024-09-16T02:05:30.543418+0000","last_clean":"2024-09-16T02:05:30.543418+0000","last_became_active":"2024-09-16T02:03:37.050559+0000","last_became_peered":"2024-09-16T02:03:37.050559+0000","last_unstale":"2024-09-16T02:05:30.543418+0000","last_undegraded":"2024-09-16T02:05:30.543418+0000","last_fullsized":"2024-09-16T02:05:30.543418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:20.067954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:34.271880+0000","last_change":"2024-09-16T02:03:34.740286+0000","last_active":"2024-09-16T02:05:34.271880+0000","last_peered":"2024-09-16T02:05:34.271880+0000","last_clean":"2024-09-16T02:05:34.271880+0000","last_became_active":"2024-09-16T02:03:34.739892+0000","last_became_peered":"2024-09-16T02:03:34.739892+0000","last_unstale":"2024-09-16T02:05:34.271880+0000","last_undegraded":"2024-09-16T02:05:34.271880+0000","last_fullsized":"2024-09-16T02:05:34.271880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:59:14.809991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549598+0000","last_change":"2024-09-16T02:03:37.042207+0000","last_active":"2024-09-16T02:05:30.549598+0000","last_peered":"2024-09-16T02:05:30.549598+0000","last_clean":"2024-09-16T02:05:30.549598+0000","last_became_active":"2024-09-16T02:03:37.041283+0000","last_became_peered":"2024-09-16T02:03:37.041283+0000","last_unstale":"2024-09-16T02:05:30.549598+0000","last_undegraded":"2024-09-16T02:05:30.549598+0000","last_fullsized":"2024-09-16T02:05:30.549598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:45.569721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659831+0000","last_change":"2024-09-16T02:03:34.737834+0000","last_active":"2024-09-16T02:05:30.659831+0000","last_peered":"2024-09-16T02:05:30.659831+0000","last_clean":"2024-09-16T02:05:30.659831+0000","last_became_active":"2024-09-16T02:03:34.736706+0000","last_became_peered":"2024-09-16T02:03:34.736706+0000","last_unstale":"2024-09-16T02:05:30.659831+0000","last_undegraded":"2024-09-16T02:05:30.659831+0000","last_fullsized":"2024-09-16T02:05:30.659831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:27.037351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.546182+0000","last_change":"2024-09-16T02:03:37.060888+0000","last_active":"2024-09-16T02:05:30.546182+0000","last_peered":"2024-09-16T02:05:30.546182+0000","last_clean":"2024-09-16T02:05:30.546182+0000","last_became_active":"2024-09-16T02:03:37.060458+0000","last_became_peered":"2024-09-16T02:03:37.060458+0000","last_unstale":"2024-09-16T02:05:30.546182+0000","last_undegraded":"2024-09-16T02:05:30.546182+0000","last_fullsized":"2024-09-16T02:05:30.546182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:37.425376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548446+0000","last_change":"2024-09-16T02:03:34.742298+0000","last_active":"2024-09-16T02:05:30.548446+0000","last_peered":"2024-09-16T02:05:30.548446+0000","last_clean":"2024-09-16T02:05:30.548446+0000","last_became_active":"2024-09-16T02:03:34.741960+0000","last_became_peered":"2024-09-16T02:03:34.741960+0000","last_unstale":"2024-09-16T02:05:30.548446+0000","last_undegraded":"2024-09-16T02:05:30.548446+0000","last_fullsized":"2024-09-16T02:05:30.548446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:12.832207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545922+0000","last_change":"2024-09-16T02:03:37.060553+0000","last_active":"2024-09-16T02:05:30.545922+0000","last_peered":"2024-09-16T02:05:30.545922+0000","last_clean":"2024-09-16T02:05:30.545922+0000","last_became_active":"2024-09-16T02:03:37.059961+0000","last_became_peered":"2024-09-16T02:03:37.059961+0000","last_unstale":"2024-09-16T02:05:30.545922+0000","last_undegraded":"2024-09-16T02:05:30.545922+0000","last_fullsized":"2024-09-16T02:05:30.545922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:22.904190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545737+0000","last_change":"2024-09-16T02:03:34.747828+0000","last_active":"2024-09-16T02:05:30.545737+0000","last_peered":"2024-09-16T02:05:30.545737+0000","last_clean":"2024-09-16T02:05:30.545737+0000","last_became_active":"2024-09-16T02:03:34.747548+0000","last_became_peered":"2024-09-16T02:03:34.747548+0000","last_unstale":"2024-09-16T02:05:30.545737+0000","last_undegraded":"2024-09-16T02:05:30.545737+0000","last_fullsized":"2024-09-16T02:05:30.545737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:49.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543830+0000","last_change":"2024-09-16T02:03:37.051477+0000","last_active":"2024-09-16T02:05:30.543830+0000","last_peered":"2024-09-16T02:05:30.543830+0000","last_clean":"2024-09-16T02:05:30.543830+0000","last_became_active":"2024-09-16T02:03:37.048063+0000","last_became_peered":"2024-09-16T02:03:37.048063+0000","last_unstale":"2024-09-16T02:05:30.543830+0000","last_undegraded":"2024-09-16T02:05:30.543830+0000","last_fullsized":"2024-09-16T02:05:30.543830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:15.973972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'14","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:31.053974+0000","last_change":"2024-09-16T02:03:34.742935+0000","last_active":"2024-09-16T02:05:31.053974+0000","last_peered":"2024-09-16T02:05:31.053974+0000","last_clean":"2024-09-16T02:05:31.053974+0000","last_became_active":"2024-09-16T02:03:34.742611+0000","last_became_peered":"2024-09-16T02:03:34.742611+0000","last_unstale":"2024-09-16T02:05:31.053974+0000","last_undegraded":"2024-09-16T02:05:31.053974+0000","last_fullsized":"2024-09-16T02:05:31.053974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:36.695220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926955+0000","last_change":"2024-09-16T02:03:37.054926+0000","last_active":"2024-09-16T02:05:30.926955+0000","last_peered":"2024-09-16T02:05:30.926955+0000","last_clean":"2024-09-16T02:05:30.926955+0000","last_became_active":"2024-09-16T02:03:37.054483+0000","last_became_peered":"2024-09-16T02:03:37.054483+0000","last_unstale":"2024-09-16T02:05:30.926955+0000","last_undegraded":"2024-09-16T02:05:30.926955+0000","last_fullsized":"2024-09-16T02:05:30.926955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545961+0000","last_change":"2024-09-16T02:03:34.739611+0000","last_active":"2024-09-16T02:05:30.545961+0000","last_peered":"2024-09-16T02:05:30.545961+0000","last_clean":"2024-09-16T02:05:30.545961+0000","last_became_active":"2024-09-16T02:03:34.738090+0000","last_became_peered":"2024-09-16T02:03:34.738090+0000","last_unstale":"2024-09-16T02:05:30.545961+0000","last_undegraded":"2024-09-16T02:05:30.545961+0000","last_fullsized":"2024-09-16T02:05:30.545961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:47.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847671+0000","last_change":"2024-09-16T02:03:37.053020+0000","last_active":"2024-09-16T02:05:30.847671+0000","last_peered":"2024-09-16T02:05:30.847671+0000","last_clean":"2024-09-16T02:05:30.847671+0000","last_became_active":"2024-09-16T02:03:37.052365+0000","last_became_peered":"2024-09-16T02:03:37.052365+0000","last_unstale":"2024-09-16T02:05:30.847671+0000","last_undegraded":"2024-09-16T02:05:30.847671+0000","last_fullsized":"2024-09-16T02:05:30.847671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:54.965028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549356+0000","last_change":"2024-09-16T02:03:34.740726+0000","last_active":"2024-09-16T02:05:30.549356+0000","last_peered":"2024-09-16T02:05:30.549356+0000","last_clean":"2024-09-16T02:05:30.549356+0000","last_became_active":"2024-09-16T02:03:34.740402+0000","last_became_peered":"2024-09-16T02:03:34.740402+0000","last_unstale":"2024-09-16T02:05:30.549356+0000","last_undegraded":"2024-09-16T02:05:30.549356+0000","last_fullsized":"2024-09-16T02:05:30.549356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:05.761646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545210+0000","last_change":"2024-09-16T02:03:37.044862+0000","last_active":"2024-09-16T02:05:30.545210+0000","last_peered":"2024-09-16T02:05:30.545210+0000","last_clean":"2024-09-16T02:05:30.545210+0000","last_became_active":"2024-09-16T02:03:37.044406+0000","last_became_peered":"2024-09-16T02:03:37.044406+0000","last_unstale":"2024-09-16T02:05:30.545210+0000","last_undegraded":"2024-09-16T02:05:30.545210+0000","last_fullsized":"2024-09-16T02:05:30.545210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:03.326380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926544+0000","last_change":"2024-09-16T02:03:34.745505+0000","last_active":"2024-09-16T02:05:30.926544+0000","last_peered":"2024-09-16T02:05:30.926544+0000","last_clean":"2024-09-16T02:05:30.926544+0000","last_became_active":"2024-09-16T02:03:34.745151+0000","last_became_peered":"2024-09-16T02:03:34.745151+0000","last_unstale":"2024-09-16T02:05:30.926544+0000","last_undegraded":"2024-09-16T02:05:30.926544+0000","last_fullsized":"2024-09-16T02:05:30.926544+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:57.401176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547672+0000","last_change":"2024-09-16T02:03:37.046692+0000","last_active":"2024-09-16T02:05:30.547672+0000","last_peered":"2024-09-16T02:05:30.547672+0000","last_clean":"2024-09-16T02:05:30.547672+0000","last_became_active":"2024-09-16T02:03:37.046302+0000","last_became_peered":"2024-09-16T02:03:37.046302+0000","last_unstale":"2024-09-16T02:05:30.547672+0000","last_undegraded":"2024-09-16T02:05:30.547672+0000","last_fullsized":"2024-09-16T02:05:30.547672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:47.727431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544513+0000","last_change":"2024-09-16T02:03:34.739847+0000","last_active":"2024-09-16T02:05:30.544513+0000","last_peered":"2024-09-16T02:05:30.544513+0000","last_clean":"2024-09-16T02:05:30.544513+0000","last_became_active":"2024-09-16T02:03:34.739514+0000","last_became_peered":"2024-09-16T02:03:34.739514+0000","last_unstale":"2024-09-16T02:05:30.544513+0000","last_undegraded":"2024-09-16T02:05:30.544513+0000","last_fullsized":"2024-09-16T02:05:30.544513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:44.617127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926659+0000","last_change":"2024-09-16T02:03:37.054910+0000","last_active":"2024-09-16T02:05:30.926659+0000","last_peered":"2024-09-16T02:05:30.926659+0000","last_clean":"2024-09-16T02:05:30.926659+0000","last_became_active":"2024-09-16T02:03:37.053852+0000","last_became_peered":"2024-09-16T02:03:37.053852+0000","last_unstale":"2024-09-16T02:05:30.926659+0000","last_undegraded":"2024-09-16T02:05:30.926659+0000","last_fullsized":"2024-09-16T02:05:30.926659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:29.042154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545928+0000","last_change":"2024-09-16T02:03:34.743027+0000","last_active":"2024-09-16T02:05:30.545928+0000","last_peered":"2024-09-16T02:05:30.545928+0000","last_clean":"2024-09-16T02:05:30.545928+0000","last_became_active":"2024-09-16T02:03:34.742658+0000","last_became_peered":"2024-09-16T02:03:34.742658+0000","last_unstale":"2024-09-16T02:05:30.545928+0000","last_undegraded":"2024-09-16T02:05:30.545928+0000","last_fullsized":"2024-09-16T02:05:30.545928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:36.779568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659557+0000","last_change":"2024-09-16T02:03:37.052783+0000","last_active":"2024-09-16T02:05:30.659557+0000","last_peered":"2024-09-16T02:05:30.659557+0000","last_clean":"2024-09-16T02:05:30.659557+0000","last_became_active":"2024-09-16T02:03:37.052409+0000","last_became_peered":"2024-09-16T02:03:37.052409+0000","last_unstale":"2024-09-16T02:05:30.659557+0000","last_undegraded":"2024-09-16T02:05:30.659557+0000","last_fullsized":"2024-09-16T02:05:30.659557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:52.667379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.551560+0000","last_change":"2024-09-16T02:05:30.551560+0000","last_active":"2024-09-16T02:05:30.551560+0000","last_peered":"2024-09-16T02:05:30.551560+0000","last_clean":"2024-09-16T02:05:30.551560+0000","last_became_active":"2024-09-16T02:05:30.551063+0000","last_became_peered":"2024-09-16T02:05:30.551063+0000","last_unstale":"2024-09-16T02:05:30.551560+0000","last_undegraded":"2024-09-16T02:05:30.551560+0000","last_fullsized":"2024-09-16T02:05:30.551560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:39.081149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545662+0000","last_change":"2024-09-16T02:03:34.744240+0000","last_active":"2024-09-16T02:05:30.545662+0000","last_peered":"2024-09-16T02:05:30.545662+0000","last_clean":"2024-09-16T02:05:30.545662+0000","last_became_active":"2024-09-16T02:03:34.743898+0000","last_became_peered":"2024-09-16T02:03:34.743898+0000","last_unstale":"2024-09-16T02:05:30.545662+0000","last_undegraded":"2024-09-16T02:05:30.545662+0000","last_fullsized":"2024-09-16T02:05:30.545662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:37.820529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549458+0000","last_change":"2024-09-16T02:03:37.055769+0000","last_active":"2024-09-16T02:05:30.549458+0000","last_peered":"2024-09-16T02:05:30.549458+0000","last_clean":"2024-09-16T02:05:30.549458+0000","last_became_active":"2024-09-16T02:03:37.054458+0000","last_became_peered":"2024-09-16T02:03:37.054458+0000","last_unstale":"2024-09-16T02:05:30.549458+0000","last_undegraded":"2024-09-16T02:05:30.549458+0000","last_fullsized":"2024-09-16T02:05:30.549458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:30.218712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545637+0000","last_change":"2024-09-16T02:04:29.647534+0000","last_active":"2024-09-16T02:05:30.545637+0000","last_peered":"2024-09-16T02:05:30.545637+0000","last_clean":"2024-09-16T02:05:30.545637+0000","last_became_active":"2024-09-16T02:04:29.640317+0000","last_became_peered":"2024-09-16T02:04:29.640317+0000","last_unstale":"2024-09-16T02:05:30.545637+0000","last_undegraded":"2024-09-16T02:05:30.545637+0000","last_fullsized":"2024-09-16T02:05:30.545637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:14:33.490508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,6],"acting":[8,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847438+0000","last_change":"2024-09-16T02:03:37.054370+0000","last_active":"2024-09-16T02:05:30.847438+0000","last_peered":"2024-09-16T02:05:30.847438+0000","last_clean":"2024-09-16T02:05:30.847438+0000","last_became_active":"2024-09-16T02:03:37.054051+0000","last_became_peered":"2024-09-16T02:03:37.054051+0000","last_unstale":"2024-09-16T02:05:30.847438+0000","last_undegraded":"2024-09-16T02:05:30.847438+0000","last_fullsized":"2024-09-16T02:05:30.847438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:21:42.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.927087+0000","last_change":"2024-09-16T02:03:34.739696+0000","last_active":"2024-09-16T02:05:30.927087+0000","last_peered":"2024-09-16T02:05:30.927087+0000","last_clean":"2024-09-16T02:05:30.927087+0000","last_became_active":"2024-09-16T02:03:34.738867+0000","last_became_peered":"2024-09-16T02:03:34.738867+0000","last_unstale":"2024-09-16T02:05:30.927087+0000","last_undegraded":"2024-09-16T02:05:30.927087+0000","last_fullsized":"2024-09-16T02:05:30.927087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:30.753970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926979+0000","last_change":"2024-09-16T02:03:37.058118+0000","last_active":"2024-09-16T02:05:30.926979+0000","last_peered":"2024-09-16T02:05:30.926979+0000","last_clean":"2024-09-16T02:05:30.926979+0000","last_became_active":"2024-09-16T02:03:37.057391+0000","last_became_peered":"2024-09-16T02:03:37.057391+0000","last_unstale":"2024-09-16T02:05:30.926979+0000","last_undegraded":"2024-09-16T02:05:30.926979+0000","last_fullsized":"2024-09-16T02:05:30.926979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:39.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548774+0000","last_change":"2024-09-16T02:03:34.742184+0000","last_active":"2024-09-16T02:05:30.548774+0000","last_peered":"2024-09-16T02:05:30.548774+0000","last_clean":"2024-09-16T02:05:30.548774+0000","last_became_active":"2024-09-16T02:03:34.741838+0000","last_became_peered":"2024-09-16T02:03:34.741838+0000","last_unstale":"2024-09-16T02:05:30.548774+0000","last_undegraded":"2024-09-16T02:05:30.548774+0000","last_fullsized":"2024-09-16T02:05:30.548774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:57.338380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65313,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":659,"num_read_kb":404,"num_write":181,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1149,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":207864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678054,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":440447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906985,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1597440,"data_stored":1092985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037807,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":440571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266742,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":440379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495674,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712488,"statfs":{"total":95995035648,"available":95961587712,"internally_reserved":0,"allocated":1511424,"data_stored":1033802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659199,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":905216,"data_stored":439960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888131,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1519616,"data_stored":1032226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051657,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":442398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280591,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":921600,"data_stored":440271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476820,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":506571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":970752,"data_stored":500099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":925696,"data_stored":441231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:05:37.239 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-16T02:05:37.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T02:05:37.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.429+0000 7fb3a4800640 1 -- 172.21.15.103:0/1716968173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c076230 msgr2=0x7fb39c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:37.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.429+0000 7fb3a4800640 1 --2- 172.21.15.103:0/1716968173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c076230 0x7fb39c074680 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb390009900 tx=0x7fb39002a410 comp rx=0 tx=0).stop 2024-09-16T02:05:37.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.429+0000 7fb3a4800640 1 -- 172.21.15.103:0/1716968173 shutdown_connections 2024-09-16T02:05:37.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.429+0000 7fb3a4800640 1 --2- 172.21.15.103:0/1716968173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb39c109980 0x7fb39c10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.429+0000 7fb3a4800640 1 --2- 172.21.15.103:0/1716968173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb39c074c50 0x7fb39c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.429+0000 7fb3a4800640 1 --2- 172.21.15.103:0/1716968173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c076230 0x7fb39c074680 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.429+0000 7fb3a4800640 1 -- 172.21.15.103:0/1716968173 >> 172.21.15.103:0/1716968173 conn(0x7fb39c0fd2b0 msgr2=0x7fb39c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:37.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.429+0000 7fb3a4800640 1 -- 172.21.15.103:0/1716968173 shutdown_connections 2024-09-16T02:05:37.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.429+0000 7fb3a4800640 1 -- 172.21.15.103:0/1716968173 wait complete. 2024-09-16T02:05:37.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.430+0000 7fb3a4800640 1 Processor -- start 2024-09-16T02:05:37.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.430+0000 7fb3a4800640 1 -- start start 2024-09-16T02:05:37.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.430+0000 7fb3a4800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb39c074c50 0x7fb39c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:37.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.430+0000 7fb3a4800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb39c076230 0x7fb39c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:37.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.430+0000 7fb3a4800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c109980 0x7fb39c1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.430+0000 7fb3a4800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c10f9b0 con 0x7fb39c109980 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.430+0000 7fb3a4800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c10f830 con 0x7fb39c074c50 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.430+0000 7fb3a4800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c10fb30 con 0x7fb39c076230 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c109980 0x7fb39c1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c109980 0x7fb39c1a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54622/0 (socket says 172.21.15.103:54622) 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb39c076230 0x7fb39c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a2800640 1 -- 172.21.15.103:0/2628955317 learned_addr learned my addr 172.21.15.103:0/2628955317 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a1e00640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb39c074c50 0x7fb39c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a2800640 1 -- 172.21.15.103:0/2628955317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb39c076230 msgr2=0x7fb39c19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a2800640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb39c076230 0x7fb39c19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a2800640 1 -- 172.21.15.103:0/2628955317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb39c074c50 msgr2=0x7fb39c19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a2800640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb39c074c50 0x7fb39c19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a2800640 1 -- 172.21.15.103:0/2628955317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c1a1c50 con 0x7fb39c109980 2024-09-16T02:05:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a2800640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c109980 0x7fb39c1a15e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb398009fd0 tx=0x7fb39800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb396a00640 1 -- 172.21.15.103:0/2628955317 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398017070 con 0x7fb39c109980 2024-09-16T02:05:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb396a00640 1 -- 172.21.15.103:0/2628955317 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb398003e50 con 0x7fb39c109980 2024-09-16T02:05:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb396a00640 1 -- 172.21.15.103:0/2628955317 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3980106c0 con 0x7fb39c109980 2024-09-16T02:05:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.431+0000 7fb3a4800640 1 -- 172.21.15.103:0/2628955317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c1a1f40 con 0x7fb39c109980 2024-09-16T02:05:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.432+0000 7fb3a4800640 1 -- 172.21.15.103:0/2628955317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c1a23b0 con 0x7fb39c109980 2024-09-16T02:05:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.433+0000 7fb3a4800640 1 -- 172.21.15.103:0/2628955317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c108700 con 0x7fb39c109980 2024-09-16T02:05:37.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.435+0000 7fb396a00640 1 -- 172.21.15.103:0/2628955317 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fb398003970 con 0x7fb39c109980 2024-09-16T02:05:37.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.436+0000 7fb396a00640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb3840a32e0 0x7fb3840a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:37.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.436+0000 7fb396a00640 1 -- 172.21.15.103:0/2628955317 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fb3980cf810 con 0x7fb39c109980 2024-09-16T02:05:37.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.436+0000 7fb3a1e00640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb3840a32e0 0x7fb3840a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:37.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.437+0000 7fb3a1e00640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb3840a32e0 0x7fb3840a57a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb390031440 tx=0x7fb3900315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:37.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.440+0000 7fb396a00640 1 -- 172.21.15.103:0/2628955317 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb39809bd00 con 0x7fb39c109980 2024-09-16T02:05:37.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.614+0000 7fb3a4800640 1 -- 172.21.15.103:0/2628955317 --> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb39c062630 con 0x7fb3840a32e0 2024-09-16T02:05:37.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.621+0000 7fb396a00640 1 -- 172.21.15.103:0/2628955317 <== mgr.14150 v2:172.21.15.103:6800/3075264030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258899 (secure 0 0 0) 0x7fb39c062630 con 0x7fb3840a32e0 2024-09-16T02:05:37.626 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:37.628 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-16T02:05:37.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 -- 172.21.15.103:0/2628955317 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb3840a32e0 msgr2=0x7fb3840a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:37.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb3840a32e0 0x7fb3840a57a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb390031440 tx=0x7fb3900315b0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 -- 172.21.15.103:0/2628955317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c109980 msgr2=0x7fb39c1a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:37.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c109980 0x7fb39c1a15e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb398009fd0 tx=0x7fb39800b260 comp rx=0 tx=0).stop 2024-09-16T02:05:37.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 -- 172.21.15.103:0/2628955317 shutdown_connections 2024-09-16T02:05:37.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fb3840a32e0 0x7fb3840a57a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c109980 0x7fb39c1a15e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb39c076230 0x7fb39c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 --2- 172.21.15.103:0/2628955317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb39c074c50 0x7fb39c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:37.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 -- 172.21.15.103:0/2628955317 >> 172.21.15.103:0/2628955317 conn(0x7fb39c0fd2b0 msgr2=0x7fb39c1058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:37.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 -- 172.21.15.103:0/2628955317 shutdown_connections 2024-09-16T02:05:37.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:37.626+0000 7fb3a4800640 1 -- 172.21.15.103:0/2628955317 wait complete. 2024-09-16T02:05:37.644 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-16T02:05:36.478557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655915,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":785,"num_read_kb":513,"num_write":364,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181885,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":791,"ondisk_log_size":791,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370328,"kb_used_data":12688,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571496,"statfs":{"total":1151940427776,"available":1151561211904,"internally_reserved":0,"allocated":12992512,"data_stored":7250940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1153,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":25,"num_write":17,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002242"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547879+0000","last_change":"2024-09-16T02:03:37.050873+0000","last_active":"2024-09-16T02:05:30.547879+0000","last_peered":"2024-09-16T02:05:30.547879+0000","last_clean":"2024-09-16T02:05:30.547879+0000","last_became_active":"2024-09-16T02:03:37.050202+0000","last_became_peered":"2024-09-16T02:03:37.050202+0000","last_unstale":"2024-09-16T02:05:30.547879+0000","last_undegraded":"2024-09-16T02:05:30.547879+0000","last_fullsized":"2024-09-16T02:05:30.547879+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:12.971199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847625+0000","last_change":"2024-09-16T02:03:37.048034+0000","last_active":"2024-09-16T02:05:30.847625+0000","last_peered":"2024-09-16T02:05:30.847625+0000","last_clean":"2024-09-16T02:05:30.847625+0000","last_became_active":"2024-09-16T02:03:37.047216+0000","last_became_peered":"2024-09-16T02:03:37.047216+0000","last_unstale":"2024-09-16T02:05:30.847625+0000","last_undegraded":"2024-09-16T02:05:30.847625+0000","last_fullsized":"2024-09-16T02:05:30.847625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:48.580466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847483+0000","last_change":"2024-09-16T02:03:37.059287+0000","last_active":"2024-09-16T02:05:30.847483+0000","last_peered":"2024-09-16T02:05:30.847483+0000","last_clean":"2024-09-16T02:05:30.847483+0000","last_became_active":"2024-09-16T02:03:37.055270+0000","last_became_peered":"2024-09-16T02:03:37.055270+0000","last_unstale":"2024-09-16T02:05:30.847483+0000","last_undegraded":"2024-09-16T02:05:30.847483+0000","last_fullsized":"2024-09-16T02:05:30.847483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:29.430514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545655+0000","last_change":"2024-09-16T02:03:37.052473+0000","last_active":"2024-09-16T02:05:30.545655+0000","last_peered":"2024-09-16T02:05:30.545655+0000","last_clean":"2024-09-16T02:05:30.545655+0000","last_became_active":"2024-09-16T02:03:37.052015+0000","last_became_peered":"2024-09-16T02:03:37.052015+0000","last_unstale":"2024-09-16T02:05:30.545655+0000","last_undegraded":"2024-09-16T02:05:30.545655+0000","last_fullsized":"2024-09-16T02:05:30.545655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:03.964132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550684+0000","last_change":"2024-09-16T02:05:30.550684+0000","last_active":"2024-09-16T02:05:30.550684+0000","last_peered":"2024-09-16T02:05:30.550684+0000","last_clean":"2024-09-16T02:05:30.550684+0000","last_became_active":"2024-09-16T02:05:30.550245+0000","last_became_peered":"2024-09-16T02:05:30.550245+0000","last_unstale":"2024-09-16T02:05:30.550684+0000","last_undegraded":"2024-09-16T02:05:30.550684+0000","last_fullsized":"2024-09-16T02:05:30.550684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:06.367612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545873+0000","last_change":"2024-09-16T02:03:37.048912+0000","last_active":"2024-09-16T02:05:30.545873+0000","last_peered":"2024-09-16T02:05:30.545873+0000","last_clean":"2024-09-16T02:05:30.545873+0000","last_became_active":"2024-09-16T02:03:37.048577+0000","last_became_peered":"2024-09-16T02:03:37.048577+0000","last_unstale":"2024-09-16T02:05:30.545873+0000","last_undegraded":"2024-09-16T02:05:30.545873+0000","last_fullsized":"2024-09-16T02:05:30.545873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:25.599378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659580+0000","last_change":"2024-09-16T02:03:37.057427+0000","last_active":"2024-09-16T02:05:30.659580+0000","last_peered":"2024-09-16T02:05:30.659580+0000","last_clean":"2024-09-16T02:05:30.659580+0000","last_became_active":"2024-09-16T02:03:37.057111+0000","last_became_peered":"2024-09-16T02:03:37.057111+0000","last_unstale":"2024-09-16T02:05:30.659580+0000","last_undegraded":"2024-09-16T02:05:30.659580+0000","last_fullsized":"2024-09-16T02:05:30.659580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:44.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926373+0000","last_change":"2024-09-16T02:03:37.051085+0000","last_active":"2024-09-16T02:05:30.926373+0000","last_peered":"2024-09-16T02:05:30.926373+0000","last_clean":"2024-09-16T02:05:30.926373+0000","last_became_active":"2024-09-16T02:03:37.050790+0000","last_became_peered":"2024-09-16T02:03:37.050790+0000","last_unstale":"2024-09-16T02:05:30.926373+0000","last_undegraded":"2024-09-16T02:05:30.926373+0000","last_fullsized":"2024-09-16T02:05:30.926373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:18.620901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549870+0000","last_change":"2024-09-16T02:03:37.054258+0000","last_active":"2024-09-16T02:05:30.549870+0000","last_peered":"2024-09-16T02:05:30.549870+0000","last_clean":"2024-09-16T02:05:30.549870+0000","last_became_active":"2024-09-16T02:03:37.053939+0000","last_became_peered":"2024-09-16T02:03:37.053939+0000","last_unstale":"2024-09-16T02:05:30.549870+0000","last_undegraded":"2024-09-16T02:05:30.549870+0000","last_fullsized":"2024-09-16T02:05:30.549870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:11.076147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.546724+0000","last_change":"2024-09-16T02:03:37.049828+0000","last_active":"2024-09-16T02:05:30.546724+0000","last_peered":"2024-09-16T02:05:30.546724+0000","last_clean":"2024-09-16T02:05:30.546724+0000","last_became_active":"2024-09-16T02:03:37.049577+0000","last_became_peered":"2024-09-16T02:03:37.049577+0000","last_unstale":"2024-09-16T02:05:30.546724+0000","last_undegraded":"2024-09-16T02:05:30.546724+0000","last_fullsized":"2024-09-16T02:05:30.546724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:37.176431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659716+0000","last_change":"2024-09-16T02:03:37.053268+0000","last_active":"2024-09-16T02:05:30.659716+0000","last_peered":"2024-09-16T02:05:30.659716+0000","last_clean":"2024-09-16T02:05:30.659716+0000","last_became_active":"2024-09-16T02:03:37.052933+0000","last_became_peered":"2024-09-16T02:03:37.052933+0000","last_unstale":"2024-09-16T02:05:30.659716+0000","last_undegraded":"2024-09-16T02:05:30.659716+0000","last_fullsized":"2024-09-16T02:05:30.659716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:32.003728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545762+0000","last_change":"2024-09-16T02:03:37.046339+0000","last_active":"2024-09-16T02:05:30.545762+0000","last_peered":"2024-09-16T02:05:30.545762+0000","last_clean":"2024-09-16T02:05:30.545762+0000","last_became_active":"2024-09-16T02:03:37.043262+0000","last_became_peered":"2024-09-16T02:03:37.043262+0000","last_unstale":"2024-09-16T02:05:30.545762+0000","last_undegraded":"2024-09-16T02:05:30.545762+0000","last_fullsized":"2024-09-16T02:05:30.545762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:58.614672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550406+0000","last_change":"2024-09-16T02:05:30.550406+0000","last_active":"2024-09-16T02:05:30.550406+0000","last_peered":"2024-09-16T02:05:30.550406+0000","last_clean":"2024-09-16T02:05:30.550406+0000","last_became_active":"2024-09-16T02:05:30.549945+0000","last_became_peered":"2024-09-16T02:05:30.549945+0000","last_unstale":"2024-09-16T02:05:30.550406+0000","last_undegraded":"2024-09-16T02:05:30.550406+0000","last_fullsized":"2024-09-16T02:05:30.550406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:49.789795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847255+0000","last_change":"2024-09-16T02:03:37.048844+0000","last_active":"2024-09-16T02:05:30.847255+0000","last_peered":"2024-09-16T02:05:30.847255+0000","last_clean":"2024-09-16T02:05:30.847255+0000","last_became_active":"2024-09-16T02:03:37.048522+0000","last_became_peered":"2024-09-16T02:03:37.048522+0000","last_unstale":"2024-09-16T02:05:30.847255+0000","last_undegraded":"2024-09-16T02:05:30.847255+0000","last_fullsized":"2024-09-16T02:05:30.847255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:50.759847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847176+0000","last_change":"2024-09-16T02:03:37.043236+0000","last_active":"2024-09-16T02:05:30.847176+0000","last_peered":"2024-09-16T02:05:30.847176+0000","last_clean":"2024-09-16T02:05:30.847176+0000","last_became_active":"2024-09-16T02:03:37.042967+0000","last_became_peered":"2024-09-16T02:03:37.042967+0000","last_unstale":"2024-09-16T02:05:30.847176+0000","last_undegraded":"2024-09-16T02:05:30.847176+0000","last_fullsized":"2024-09-16T02:05:30.847176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:02.739678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549252+0000","last_change":"2024-09-16T02:03:37.052972+0000","last_active":"2024-09-16T02:05:30.549252+0000","last_peered":"2024-09-16T02:05:30.549252+0000","last_clean":"2024-09-16T02:05:30.549252+0000","last_became_active":"2024-09-16T02:03:37.052665+0000","last_became_peered":"2024-09-16T02:03:37.052665+0000","last_unstale":"2024-09-16T02:05:30.549252+0000","last_undegraded":"2024-09-16T02:05:30.549252+0000","last_fullsized":"2024-09-16T02:05:30.549252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:59.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547979+0000","last_change":"2024-09-16T02:03:37.051203+0000","last_active":"2024-09-16T02:05:30.547979+0000","last_peered":"2024-09-16T02:05:30.547979+0000","last_clean":"2024-09-16T02:05:30.547979+0000","last_became_active":"2024-09-16T02:03:37.049828+0000","last_became_peered":"2024-09-16T02:03:37.049828+0000","last_unstale":"2024-09-16T02:05:30.547979+0000","last_undegraded":"2024-09-16T02:05:30.547979+0000","last_fullsized":"2024-09-16T02:05:30.547979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:47.566594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548575+0000","last_change":"2024-09-16T02:03:37.056903+0000","last_active":"2024-09-16T02:05:30.548575+0000","last_peered":"2024-09-16T02:05:30.548575+0000","last_clean":"2024-09-16T02:05:30.548575+0000","last_became_active":"2024-09-16T02:03:37.056579+0000","last_became_peered":"2024-09-16T02:03:37.056579+0000","last_unstale":"2024-09-16T02:05:30.548575+0000","last_undegraded":"2024-09-16T02:05:30.548575+0000","last_fullsized":"2024-09-16T02:05:30.548575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:23.652294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544817+0000","last_change":"2024-09-16T02:03:37.054413+0000","last_active":"2024-09-16T02:05:30.544817+0000","last_peered":"2024-09-16T02:05:30.544817+0000","last_clean":"2024-09-16T02:05:30.544817+0000","last_became_active":"2024-09-16T02:03:37.054040+0000","last_became_peered":"2024-09-16T02:03:37.054040+0000","last_unstale":"2024-09-16T02:05:30.544817+0000","last_undegraded":"2024-09-16T02:05:30.544817+0000","last_fullsized":"2024-09-16T02:05:30.544817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:04.517313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547502+0000","last_change":"2024-09-16T02:03:37.053454+0000","last_active":"2024-09-16T02:05:30.547502+0000","last_peered":"2024-09-16T02:05:30.547502+0000","last_clean":"2024-09-16T02:05:30.547502+0000","last_became_active":"2024-09-16T02:03:37.053123+0000","last_became_peered":"2024-09-16T02:03:37.053123+0000","last_unstale":"2024-09-16T02:05:30.547502+0000","last_undegraded":"2024-09-16T02:05:30.547502+0000","last_fullsized":"2024-09-16T02:05:30.547502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:52.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543738+0000","last_change":"2024-09-16T02:03:37.042772+0000","last_active":"2024-09-16T02:05:30.543738+0000","last_peered":"2024-09-16T02:05:30.543738+0000","last_clean":"2024-09-16T02:05:30.543738+0000","last_became_active":"2024-09-16T02:03:37.041832+0000","last_became_peered":"2024-09-16T02:03:37.041832+0000","last_unstale":"2024-09-16T02:05:30.543738+0000","last_undegraded":"2024-09-16T02:05:30.543738+0000","last_fullsized":"2024-09-16T02:05:30.543738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:54.436291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549491+0000","last_change":"2024-09-16T02:03:37.058484+0000","last_active":"2024-09-16T02:05:30.549491+0000","last_peered":"2024-09-16T02:05:30.549491+0000","last_clean":"2024-09-16T02:05:30.549491+0000","last_became_active":"2024-09-16T02:03:37.058047+0000","last_became_peered":"2024-09-16T02:03:37.058047+0000","last_unstale":"2024-09-16T02:05:30.549491+0000","last_undegraded":"2024-09-16T02:05:30.549491+0000","last_fullsized":"2024-09-16T02:05:30.549491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:23.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926399+0000","last_change":"2024-09-16T02:03:37.058576+0000","last_active":"2024-09-16T02:05:30.926399+0000","last_peered":"2024-09-16T02:05:30.926399+0000","last_clean":"2024-09-16T02:05:30.926399+0000","last_became_active":"2024-09-16T02:03:37.057624+0000","last_became_peered":"2024-09-16T02:03:37.057624+0000","last_unstale":"2024-09-16T02:05:30.926399+0000","last_undegraded":"2024-09-16T02:05:30.926399+0000","last_fullsized":"2024-09-16T02:05:30.926399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:21.929461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549455+0000","last_change":"2024-09-16T02:03:37.050889+0000","last_active":"2024-09-16T02:05:30.549455+0000","last_peered":"2024-09-16T02:05:30.549455+0000","last_clean":"2024-09-16T02:05:30.549455+0000","last_became_active":"2024-09-16T02:03:37.050535+0000","last_became_peered":"2024-09-16T02:03:37.050535+0000","last_unstale":"2024-09-16T02:05:30.549455+0000","last_undegraded":"2024-09-16T02:05:30.549455+0000","last_fullsized":"2024-09-16T02:05:30.549455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:46.955962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926844+0000","last_change":"2024-09-16T02:03:37.053865+0000","last_active":"2024-09-16T02:05:30.926844+0000","last_peered":"2024-09-16T02:05:30.926844+0000","last_clean":"2024-09-16T02:05:30.926844+0000","last_became_active":"2024-09-16T02:03:37.053563+0000","last_became_peered":"2024-09-16T02:03:37.053563+0000","last_unstale":"2024-09-16T02:05:30.926844+0000","last_undegraded":"2024-09-16T02:05:30.926844+0000","last_fullsized":"2024-09-16T02:05:30.926844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:24.109471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545076+0000","last_change":"2024-09-16T02:03:37.054681+0000","last_active":"2024-09-16T02:05:30.545076+0000","last_peered":"2024-09-16T02:05:30.545076+0000","last_clean":"2024-09-16T02:05:30.545076+0000","last_became_active":"2024-09-16T02:03:37.054324+0000","last_became_peered":"2024-09-16T02:03:37.054324+0000","last_unstale":"2024-09-16T02:05:30.545076+0000","last_undegraded":"2024-09-16T02:05:30.545076+0000","last_fullsized":"2024-09-16T02:05:30.545076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:20.472520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547733+0000","last_change":"2024-09-16T02:03:37.047713+0000","last_active":"2024-09-16T02:05:30.547733+0000","last_peered":"2024-09-16T02:05:30.547733+0000","last_clean":"2024-09-16T02:05:30.547733+0000","last_became_active":"2024-09-16T02:03:37.047353+0000","last_became_peered":"2024-09-16T02:03:37.047353+0000","last_unstale":"2024-09-16T02:05:30.547733+0000","last_undegraded":"2024-09-16T02:05:30.547733+0000","last_fullsized":"2024-09-16T02:05:30.547733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:05.868909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543731+0000","last_change":"2024-09-16T02:03:37.038931+0000","last_active":"2024-09-16T02:05:30.543731+0000","last_peered":"2024-09-16T02:05:30.543731+0000","last_clean":"2024-09-16T02:05:30.543731+0000","last_became_active":"2024-09-16T02:03:37.037234+0000","last_became_peered":"2024-09-16T02:03:37.037234+0000","last_unstale":"2024-09-16T02:05:30.543731+0000","last_undegraded":"2024-09-16T02:05:30.543731+0000","last_fullsized":"2024-09-16T02:05:30.543731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:54.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847518+0000","last_change":"2024-09-16T02:04:29.634327+0000","last_active":"2024-09-16T02:05:30.847518+0000","last_peered":"2024-09-16T02:05:30.847518+0000","last_clean":"2024-09-16T02:05:30.847518+0000","last_became_active":"2024-09-16T02:04:29.633710+0000","last_became_peered":"2024-09-16T02:04:29.633710+0000","last_unstale":"2024-09-16T02:05:30.847518+0000","last_undegraded":"2024-09-16T02:05:30.847518+0000","last_fullsized":"2024-09-16T02:05:30.847518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:55:03.574092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544503+0000","last_change":"2024-09-16T02:03:37.057688+0000","last_active":"2024-09-16T02:05:30.544503+0000","last_peered":"2024-09-16T02:05:30.544503+0000","last_clean":"2024-09-16T02:05:30.544503+0000","last_became_active":"2024-09-16T02:03:37.057347+0000","last_became_peered":"2024-09-16T02:03:37.057347+0000","last_unstale":"2024-09-16T02:05:30.544503+0000","last_undegraded":"2024-09-16T02:05:30.544503+0000","last_fullsized":"2024-09-16T02:05:30.544503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:30.595220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544749+0000","last_change":"2024-09-16T02:04:29.640455+0000","last_active":"2024-09-16T02:05:30.544749+0000","last_peered":"2024-09-16T02:05:30.544749+0000","last_clean":"2024-09-16T02:05:30.544749+0000","last_became_active":"2024-09-16T02:04:29.640083+0000","last_became_peered":"2024-09-16T02:04:29.640083+0000","last_unstale":"2024-09-16T02:05:30.544749+0000","last_undegraded":"2024-09-16T02:05:30.544749+0000","last_fullsized":"2024-09-16T02:05:30.544749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:46.191484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547527+0000","last_change":"2024-09-16T02:03:37.043596+0000","last_active":"2024-09-16T02:05:30.547527+0000","last_peered":"2024-09-16T02:05:30.547527+0000","last_clean":"2024-09-16T02:05:30.547527+0000","last_became_active":"2024-09-16T02:03:37.043261+0000","last_became_peered":"2024-09-16T02:03:37.043261+0000","last_unstale":"2024-09-16T02:05:30.547527+0000","last_undegraded":"2024-09-16T02:05:30.547527+0000","last_fullsized":"2024-09-16T02:05:30.547527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:03.833126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545258+0000","last_change":"2024-09-16T02:03:37.043018+0000","last_active":"2024-09-16T02:05:30.545258+0000","last_peered":"2024-09-16T02:05:30.545258+0000","last_clean":"2024-09-16T02:05:30.545258+0000","last_became_active":"2024-09-16T02:03:37.042437+0000","last_became_peered":"2024-09-16T02:03:37.042437+0000","last_unstale":"2024-09-16T02:05:30.545258+0000","last_undegraded":"2024-09-16T02:05:30.545258+0000","last_fullsized":"2024-09-16T02:05:30.545258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.193852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'7","reported_seq":153,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.739530+0000","last_change":"2024-09-16T02:04:29.645481+0000","last_active":"2024-09-16T02:05:30.739530+0000","last_peered":"2024-09-16T02:05:30.739530+0000","last_clean":"2024-09-16T02:05:30.739530+0000","last_became_active":"2024-09-16T02:04:29.640089+0000","last_became_peered":"2024-09-16T02:04:29.640089+0000","last_unstale":"2024-09-16T02:05:30.739530+0000","last_undegraded":"2024-09-16T02:05:30.739530+0000","last_fullsized":"2024-09-16T02:05:30.739530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:02.431443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549918+0000","last_change":"2024-09-16T02:03:37.052664+0000","last_active":"2024-09-16T02:05:30.549918+0000","last_peered":"2024-09-16T02:05:30.549918+0000","last_clean":"2024-09-16T02:05:30.549918+0000","last_became_active":"2024-09-16T02:03:37.051310+0000","last_became_peered":"2024-09-16T02:03:37.051310+0000","last_unstale":"2024-09-16T02:05:30.549918+0000","last_undegraded":"2024-09-16T02:05:30.549918+0000","last_fullsized":"2024-09-16T02:05:30.549918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:57.979053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547503+0000","last_change":"2024-09-16T02:04:29.654377+0000","last_active":"2024-09-16T02:05:30.547503+0000","last_peered":"2024-09-16T02:05:30.547503+0000","last_clean":"2024-09-16T02:05:30.547503+0000","last_became_active":"2024-09-16T02:04:29.635187+0000","last_became_peered":"2024-09-16T02:04:29.635187+0000","last_unstale":"2024-09-16T02:05:30.547503+0000","last_undegraded":"2024-09-16T02:05:30.547503+0000","last_fullsized":"2024-09-16T02:05:30.547503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:28.474841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,6],"acting":[11,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543512+0000","last_change":"2024-09-16T02:03:34.745325+0000","last_active":"2024-09-16T02:05:30.543512+0000","last_peered":"2024-09-16T02:05:30.543512+0000","last_clean":"2024-09-16T02:05:30.543512+0000","last_became_active":"2024-09-16T02:03:34.744730+0000","last_became_peered":"2024-09-16T02:03:34.744730+0000","last_unstale":"2024-09-16T02:05:30.543512+0000","last_undegraded":"2024-09-16T02:05:30.543512+0000","last_fullsized":"2024-09-16T02:05:30.543512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926402+0000","last_change":"2024-09-16T02:03:37.058973+0000","last_active":"2024-09-16T02:05:30.926402+0000","last_peered":"2024-09-16T02:05:30.926402+0000","last_clean":"2024-09-16T02:05:30.926402+0000","last_became_active":"2024-09-16T02:03:37.058685+0000","last_became_peered":"2024-09-16T02:03:37.058685+0000","last_unstale":"2024-09-16T02:05:30.926402+0000","last_undegraded":"2024-09-16T02:05:30.926402+0000","last_fullsized":"2024-09-16T02:05:30.926402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:46.030440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549707+0000","last_change":"2024-09-16T02:03:34.737842+0000","last_active":"2024-09-16T02:05:30.549707+0000","last_peered":"2024-09-16T02:05:30.549707+0000","last_clean":"2024-09-16T02:05:30.549707+0000","last_became_active":"2024-09-16T02:03:34.737391+0000","last_became_peered":"2024-09-16T02:03:34.737391+0000","last_unstale":"2024-09-16T02:05:30.549707+0000","last_undegraded":"2024-09-16T02:05:30.549707+0000","last_fullsized":"2024-09-16T02:05:30.549707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:39.214180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548437+0000","last_change":"2024-09-16T02:03:37.049705+0000","last_active":"2024-09-16T02:05:30.548437+0000","last_peered":"2024-09-16T02:05:30.548437+0000","last_clean":"2024-09-16T02:05:30.548437+0000","last_became_active":"2024-09-16T02:03:37.049392+0000","last_became_peered":"2024-09-16T02:03:37.049392+0000","last_unstale":"2024-09-16T02:05:30.548437+0000","last_undegraded":"2024-09-16T02:05:30.548437+0000","last_fullsized":"2024-09-16T02:05:30.548437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.932391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:31.740755+0000","last_change":"2024-09-16T02:03:34.742268+0000","last_active":"2024-09-16T02:05:31.740755+0000","last_peered":"2024-09-16T02:05:31.740755+0000","last_clean":"2024-09-16T02:05:31.740755+0000","last_became_active":"2024-09-16T02:03:34.741934+0000","last_became_peered":"2024-09-16T02:03:34.741934+0000","last_unstale":"2024-09-16T02:05:31.740755+0000","last_undegraded":"2024-09-16T02:05:31.740755+0000","last_fullsized":"2024-09-16T02:05:31.740755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:30.301552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549786+0000","last_change":"2024-09-16T02:03:37.059119+0000","last_active":"2024-09-16T02:05:30.549786+0000","last_peered":"2024-09-16T02:05:30.549786+0000","last_clean":"2024-09-16T02:05:30.549786+0000","last_became_active":"2024-09-16T02:03:37.058738+0000","last_became_peered":"2024-09-16T02:03:37.058738+0000","last_unstale":"2024-09-16T02:05:30.549786+0000","last_undegraded":"2024-09-16T02:05:30.549786+0000","last_fullsized":"2024-09-16T02:05:30.549786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.496053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659981+0000","last_change":"2024-09-16T02:04:29.642379+0000","last_active":"2024-09-16T02:05:30.659981+0000","last_peered":"2024-09-16T02:05:30.659981+0000","last_clean":"2024-09-16T02:05:30.659981+0000","last_became_active":"2024-09-16T02:04:29.638587+0000","last_became_peered":"2024-09-16T02:04:29.638587+0000","last_unstale":"2024-09-16T02:05:30.659981+0000","last_undegraded":"2024-09-16T02:05:30.659981+0000","last_fullsized":"2024-09-16T02:05:30.659981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:50.268223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847445+0000","last_change":"2024-09-16T02:03:37.050349+0000","last_active":"2024-09-16T02:05:30.847445+0000","last_peered":"2024-09-16T02:05:30.847445+0000","last_clean":"2024-09-16T02:05:30.847445+0000","last_became_active":"2024-09-16T02:03:37.049978+0000","last_became_peered":"2024-09-16T02:03:37.049978+0000","last_unstale":"2024-09-16T02:05:30.847445+0000","last_undegraded":"2024-09-16T02:05:30.847445+0000","last_fullsized":"2024-09-16T02:05:30.847445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:10:45.119683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550509+0000","last_change":"2024-09-16T02:03:34.740246+0000","last_active":"2024-09-16T02:05:30.550509+0000","last_peered":"2024-09-16T02:05:30.550509+0000","last_clean":"2024-09-16T02:05:30.550509+0000","last_became_active":"2024-09-16T02:03:34.739908+0000","last_became_peered":"2024-09-16T02:03:34.739908+0000","last_unstale":"2024-09-16T02:05:30.550509+0000","last_undegraded":"2024-09-16T02:05:30.550509+0000","last_fullsized":"2024-09-16T02:05:30.550509+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:35.299733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549580+0000","last_change":"2024-09-16T02:03:37.056562+0000","last_active":"2024-09-16T02:05:30.549580+0000","last_peered":"2024-09-16T02:05:30.549580+0000","last_clean":"2024-09-16T02:05:30.549580+0000","last_became_active":"2024-09-16T02:03:37.056226+0000","last_became_peered":"2024-09-16T02:03:37.056226+0000","last_unstale":"2024-09-16T02:05:30.549580+0000","last_undegraded":"2024-09-16T02:05:30.549580+0000","last_fullsized":"2024-09-16T02:05:30.549580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.741538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":279,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:32.742423+0000","last_change":"2024-09-16T02:03:34.736403+0000","last_active":"2024-09-16T02:05:32.742423+0000","last_peered":"2024-09-16T02:05:32.742423+0000","last_clean":"2024-09-16T02:05:32.742423+0000","last_became_active":"2024-09-16T02:03:34.736074+0000","last_became_peered":"2024-09-16T02:03:34.736074+0000","last_unstale":"2024-09-16T02:05:32.742423+0000","last_undegraded":"2024-09-16T02:05:32.742423+0000","last_fullsized":"2024-09-16T02:05:32.742423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:34.769862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60324,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":169,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545616+0000","last_change":"2024-09-16T02:03:37.046803+0000","last_active":"2024-09-16T02:05:30.545616+0000","last_peered":"2024-09-16T02:05:30.545616+0000","last_clean":"2024-09-16T02:05:30.545616+0000","last_became_active":"2024-09-16T02:03:37.046459+0000","last_became_peered":"2024-09-16T02:03:37.046459+0000","last_unstale":"2024-09-16T02:05:30.545616+0000","last_undegraded":"2024-09-16T02:05:30.545616+0000","last_fullsized":"2024-09-16T02:05:30.545616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:55:38.460795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544547+0000","last_change":"2024-09-16T02:03:34.739016+0000","last_active":"2024-09-16T02:05:30.544547+0000","last_peered":"2024-09-16T02:05:30.544547+0000","last_clean":"2024-09-16T02:05:30.544547+0000","last_became_active":"2024-09-16T02:03:34.738583+0000","last_became_peered":"2024-09-16T02:03:34.738583+0000","last_unstale":"2024-09-16T02:05:30.544547+0000","last_undegraded":"2024-09-16T02:05:30.544547+0000","last_fullsized":"2024-09-16T02:05:30.544547+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:08.255832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543487+0000","last_change":"2024-09-16T02:03:37.053353+0000","last_active":"2024-09-16T02:05:30.543487+0000","last_peered":"2024-09-16T02:05:30.543487+0000","last_clean":"2024-09-16T02:05:30.543487+0000","last_became_active":"2024-09-16T02:03:37.051140+0000","last_became_peered":"2024-09-16T02:03:37.051140+0000","last_unstale":"2024-09-16T02:05:30.543487+0000","last_undegraded":"2024-09-16T02:05:30.543487+0000","last_fullsized":"2024-09-16T02:05:30.543487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:34.636338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549839+0000","last_change":"2024-09-16T02:03:34.742111+0000","last_active":"2024-09-16T02:05:30.549839+0000","last_peered":"2024-09-16T02:05:30.549839+0000","last_clean":"2024-09-16T02:05:30.549839+0000","last_became_active":"2024-09-16T02:03:34.741816+0000","last_became_peered":"2024-09-16T02:03:34.741816+0000","last_unstale":"2024-09-16T02:05:30.549839+0000","last_undegraded":"2024-09-16T02:05:30.549839+0000","last_fullsized":"2024-09-16T02:05:30.549839+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:36.861995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659571+0000","last_change":"2024-09-16T02:03:37.047152+0000","last_active":"2024-09-16T02:05:30.659571+0000","last_peered":"2024-09-16T02:05:30.659571+0000","last_clean":"2024-09-16T02:05:30.659571+0000","last_became_active":"2024-09-16T02:03:37.046675+0000","last_became_peered":"2024-09-16T02:03:37.046675+0000","last_unstale":"2024-09-16T02:05:30.659571+0000","last_undegraded":"2024-09-16T02:05:30.659571+0000","last_fullsized":"2024-09-16T02:05:30.659571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:58.236519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":142,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:31.267665+0000","last_change":"2024-09-16T02:03:34.738487+0000","last_active":"2024-09-16T02:05:31.267665+0000","last_peered":"2024-09-16T02:05:31.267665+0000","last_clean":"2024-09-16T02:05:31.267665+0000","last_became_active":"2024-09-16T02:03:34.737061+0000","last_became_peered":"2024-09-16T02:03:34.737061+0000","last_unstale":"2024-09-16T02:05:31.267665+0000","last_undegraded":"2024-09-16T02:05:31.267665+0000","last_fullsized":"2024-09-16T02:05:31.267665+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:24.231640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1433,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659353+0000","last_change":"2024-09-16T02:03:37.044260+0000","last_active":"2024-09-16T02:05:30.659353+0000","last_peered":"2024-09-16T02:05:30.659353+0000","last_clean":"2024-09-16T02:05:30.659353+0000","last_became_active":"2024-09-16T02:03:37.043932+0000","last_became_peered":"2024-09-16T02:03:37.043932+0000","last_unstale":"2024-09-16T02:05:30.659353+0000","last_undegraded":"2024-09-16T02:05:30.659353+0000","last_fullsized":"2024-09-16T02:05:30.659353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:55.718155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547422+0000","last_change":"2024-09-16T02:04:29.651235+0000","last_active":"2024-09-16T02:05:30.547422+0000","last_peered":"2024-09-16T02:05:30.547422+0000","last_clean":"2024-09-16T02:05:30.547422+0000","last_became_active":"2024-09-16T02:04:29.633999+0000","last_became_peered":"2024-09-16T02:04:29.633999+0000","last_unstale":"2024-09-16T02:05:30.547422+0000","last_undegraded":"2024-09-16T02:05:30.547422+0000","last_fullsized":"2024-09-16T02:05:30.547422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.616990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549533+0000","last_change":"2024-09-16T02:03:37.053040+0000","last_active":"2024-09-16T02:05:30.549533+0000","last_peered":"2024-09-16T02:05:30.549533+0000","last_clean":"2024-09-16T02:05:30.549533+0000","last_became_active":"2024-09-16T02:03:37.052185+0000","last_became_peered":"2024-09-16T02:03:37.052185+0000","last_unstale":"2024-09-16T02:05:30.549533+0000","last_undegraded":"2024-09-16T02:05:30.549533+0000","last_fullsized":"2024-09-16T02:05:30.549533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:27.676678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'3","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545839+0000","last_change":"2024-09-16T02:04:29.651376+0000","last_active":"2024-09-16T02:05:30.545839+0000","last_peered":"2024-09-16T02:05:30.545839+0000","last_clean":"2024-09-16T02:05:30.545839+0000","last_became_active":"2024-09-16T02:04:29.640260+0000","last_became_peered":"2024-09-16T02:04:29.640260+0000","last_unstale":"2024-09-16T02:05:30.545839+0000","last_undegraded":"2024-09-16T02:05:30.545839+0000","last_fullsized":"2024-09-16T02:05:30.545839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:39.426183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.661732+0000","last_change":"2024-09-16T02:05:30.661732+0000","last_active":"2024-09-16T02:05:30.661732+0000","last_peered":"2024-09-16T02:05:30.661732+0000","last_clean":"2024-09-16T02:05:30.661732+0000","last_became_active":"2024-09-16T02:05:30.661261+0000","last_became_peered":"2024-09-16T02:05:30.661261+0000","last_unstale":"2024-09-16T02:05:30.661732+0000","last_undegraded":"2024-09-16T02:05:30.661732+0000","last_fullsized":"2024-09-16T02:05:30.661732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:10.510530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543828+0000","last_change":"2024-09-16T02:03:34.738516+0000","last_active":"2024-09-16T02:05:30.543828+0000","last_peered":"2024-09-16T02:05:30.543828+0000","last_clean":"2024-09-16T02:05:30.543828+0000","last_became_active":"2024-09-16T02:03:34.737701+0000","last_became_peered":"2024-09-16T02:03:34.737701+0000","last_unstale":"2024-09-16T02:05:30.543828+0000","last_undegraded":"2024-09-16T02:05:30.543828+0000","last_fullsized":"2024-09-16T02:05:30.543828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:42.101865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659922+0000","last_change":"2024-09-16T02:03:37.046820+0000","last_active":"2024-09-16T02:05:30.659922+0000","last_peered":"2024-09-16T02:05:30.659922+0000","last_clean":"2024-09-16T02:05:30.659922+0000","last_became_active":"2024-09-16T02:03:37.046082+0000","last_became_peered":"2024-09-16T02:03:37.046082+0000","last_unstale":"2024-09-16T02:05:30.659922+0000","last_undegraded":"2024-09-16T02:05:30.659922+0000","last_fullsized":"2024-09-16T02:05:30.659922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:21.016631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.550670+0000","last_change":"2024-09-16T02:03:34.741908+0000","last_active":"2024-09-16T02:05:30.550670+0000","last_peered":"2024-09-16T02:05:30.550670+0000","last_clean":"2024-09-16T02:05:30.550670+0000","last_became_active":"2024-09-16T02:03:34.739826+0000","last_became_peered":"2024-09-16T02:03:34.739826+0000","last_unstale":"2024-09-16T02:05:30.550670+0000","last_undegraded":"2024-09-16T02:05:30.550670+0000","last_fullsized":"2024-09-16T02:05:30.550670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:04.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659551+0000","last_change":"2024-09-16T02:03:37.056459+0000","last_active":"2024-09-16T02:05:30.659551+0000","last_peered":"2024-09-16T02:05:30.659551+0000","last_clean":"2024-09-16T02:05:30.659551+0000","last_became_active":"2024-09-16T02:03:37.055902+0000","last_became_peered":"2024-09-16T02:03:37.055902+0000","last_unstale":"2024-09-16T02:05:30.659551+0000","last_undegraded":"2024-09-16T02:05:30.659551+0000","last_fullsized":"2024-09-16T02:05:30.659551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:48:30.815654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544898+0000","last_change":"2024-09-16T02:03:34.741207+0000","last_active":"2024-09-16T02:05:30.544898+0000","last_peered":"2024-09-16T02:05:30.544898+0000","last_clean":"2024-09-16T02:05:30.544898+0000","last_became_active":"2024-09-16T02:03:34.740523+0000","last_became_peered":"2024-09-16T02:03:34.740523+0000","last_unstale":"2024-09-16T02:05:30.544898+0000","last_undegraded":"2024-09-16T02:05:30.544898+0000","last_fullsized":"2024-09-16T02:05:30.544898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:36.199332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659815+0000","last_change":"2024-09-16T02:03:37.052792+0000","last_active":"2024-09-16T02:05:30.659815+0000","last_peered":"2024-09-16T02:05:30.659815+0000","last_clean":"2024-09-16T02:05:30.659815+0000","last_became_active":"2024-09-16T02:03:37.052301+0000","last_became_peered":"2024-09-16T02:03:37.052301+0000","last_unstale":"2024-09-16T02:05:30.659815+0000","last_undegraded":"2024-09-16T02:05:30.659815+0000","last_fullsized":"2024-09-16T02:05:30.659815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:55.197481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":338,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659545+0000","last_change":"2024-09-16T02:02:21.095893+0000","last_active":"2024-09-16T02:05:30.659545+0000","last_peered":"2024-09-16T02:05:30.659545+0000","last_clean":"2024-09-16T02:05:30.659545+0000","last_became_active":"2024-09-16T02:02:21.095395+0000","last_became_peered":"2024-09-16T02:02:21.095395+0000","last_unstale":"2024-09-16T02:05:30.659545+0000","last_undegraded":"2024-09-16T02:05:30.659545+0000","last_fullsized":"2024-09-16T02:05:30.659545+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:58:50.954254+0000","last_clean_scrub_stamp":"2024-09-16T01:58:50.954254+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:40.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545471+0000","last_change":"2024-09-16T02:03:34.746154+0000","last_active":"2024-09-16T02:05:30.545471+0000","last_peered":"2024-09-16T02:05:30.545471+0000","last_clean":"2024-09-16T02:05:30.545471+0000","last_became_active":"2024-09-16T02:03:34.745646+0000","last_became_peered":"2024-09-16T02:03:34.745646+0000","last_unstale":"2024-09-16T02:05:30.545471+0000","last_undegraded":"2024-09-16T02:05:30.545471+0000","last_fullsized":"2024-09-16T02:05:30.545471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:06.790000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926706+0000","last_change":"2024-09-16T02:03:37.056638+0000","last_active":"2024-09-16T02:05:30.926706+0000","last_peered":"2024-09-16T02:05:30.926706+0000","last_clean":"2024-09-16T02:05:30.926706+0000","last_became_active":"2024-09-16T02:03:37.056043+0000","last_became_peered":"2024-09-16T02:03:37.056043+0000","last_unstale":"2024-09-16T02:05:30.926706+0000","last_undegraded":"2024-09-16T02:05:30.926706+0000","last_fullsized":"2024-09-16T02:05:30.926706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:35:48.353380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659548+0000","last_change":"2024-09-16T02:04:29.647682+0000","last_active":"2024-09-16T02:05:30.659548+0000","last_peered":"2024-09-16T02:05:30.659548+0000","last_clean":"2024-09-16T02:05:30.659548+0000","last_became_active":"2024-09-16T02:04:29.640065+0000","last_became_peered":"2024-09-16T02:04:29.640065+0000","last_unstale":"2024-09-16T02:05:30.659548+0000","last_undegraded":"2024-09-16T02:05:30.659548+0000","last_fullsized":"2024-09-16T02:05:30.659548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:53.765121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543418+0000","last_change":"2024-09-16T02:03:37.050895+0000","last_active":"2024-09-16T02:05:30.543418+0000","last_peered":"2024-09-16T02:05:30.543418+0000","last_clean":"2024-09-16T02:05:30.543418+0000","last_became_active":"2024-09-16T02:03:37.050559+0000","last_became_peered":"2024-09-16T02:03:37.050559+0000","last_unstale":"2024-09-16T02:05:30.543418+0000","last_undegraded":"2024-09-16T02:05:30.543418+0000","last_fullsized":"2024-09-16T02:05:30.543418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:20.067954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:34.271880+0000","last_change":"2024-09-16T02:03:34.740286+0000","last_active":"2024-09-16T02:05:34.271880+0000","last_peered":"2024-09-16T02:05:34.271880+0000","last_clean":"2024-09-16T02:05:34.271880+0000","last_became_active":"2024-09-16T02:03:34.739892+0000","last_became_peered":"2024-09-16T02:03:34.739892+0000","last_unstale":"2024-09-16T02:05:34.271880+0000","last_undegraded":"2024-09-16T02:05:34.271880+0000","last_fullsized":"2024-09-16T02:05:34.271880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:59:14.809991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549598+0000","last_change":"2024-09-16T02:03:37.042207+0000","last_active":"2024-09-16T02:05:30.549598+0000","last_peered":"2024-09-16T02:05:30.549598+0000","last_clean":"2024-09-16T02:05:30.549598+0000","last_became_active":"2024-09-16T02:03:37.041283+0000","last_became_peered":"2024-09-16T02:03:37.041283+0000","last_unstale":"2024-09-16T02:05:30.549598+0000","last_undegraded":"2024-09-16T02:05:30.549598+0000","last_fullsized":"2024-09-16T02:05:30.549598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:45.569721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659831+0000","last_change":"2024-09-16T02:03:34.737834+0000","last_active":"2024-09-16T02:05:30.659831+0000","last_peered":"2024-09-16T02:05:30.659831+0000","last_clean":"2024-09-16T02:05:30.659831+0000","last_became_active":"2024-09-16T02:03:34.736706+0000","last_became_peered":"2024-09-16T02:03:34.736706+0000","last_unstale":"2024-09-16T02:05:30.659831+0000","last_undegraded":"2024-09-16T02:05:30.659831+0000","last_fullsized":"2024-09-16T02:05:30.659831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:27.037351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.546182+0000","last_change":"2024-09-16T02:03:37.060888+0000","last_active":"2024-09-16T02:05:30.546182+0000","last_peered":"2024-09-16T02:05:30.546182+0000","last_clean":"2024-09-16T02:05:30.546182+0000","last_became_active":"2024-09-16T02:03:37.060458+0000","last_became_peered":"2024-09-16T02:03:37.060458+0000","last_unstale":"2024-09-16T02:05:30.546182+0000","last_undegraded":"2024-09-16T02:05:30.546182+0000","last_fullsized":"2024-09-16T02:05:30.546182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:37.425376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548446+0000","last_change":"2024-09-16T02:03:34.742298+0000","last_active":"2024-09-16T02:05:30.548446+0000","last_peered":"2024-09-16T02:05:30.548446+0000","last_clean":"2024-09-16T02:05:30.548446+0000","last_became_active":"2024-09-16T02:03:34.741960+0000","last_became_peered":"2024-09-16T02:03:34.741960+0000","last_unstale":"2024-09-16T02:05:30.548446+0000","last_undegraded":"2024-09-16T02:05:30.548446+0000","last_fullsized":"2024-09-16T02:05:30.548446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:12.832207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545922+0000","last_change":"2024-09-16T02:03:37.060553+0000","last_active":"2024-09-16T02:05:30.545922+0000","last_peered":"2024-09-16T02:05:30.545922+0000","last_clean":"2024-09-16T02:05:30.545922+0000","last_became_active":"2024-09-16T02:03:37.059961+0000","last_became_peered":"2024-09-16T02:03:37.059961+0000","last_unstale":"2024-09-16T02:05:30.545922+0000","last_undegraded":"2024-09-16T02:05:30.545922+0000","last_fullsized":"2024-09-16T02:05:30.545922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:22.904190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545737+0000","last_change":"2024-09-16T02:03:34.747828+0000","last_active":"2024-09-16T02:05:30.545737+0000","last_peered":"2024-09-16T02:05:30.545737+0000","last_clean":"2024-09-16T02:05:30.545737+0000","last_became_active":"2024-09-16T02:03:34.747548+0000","last_became_peered":"2024-09-16T02:03:34.747548+0000","last_unstale":"2024-09-16T02:05:30.545737+0000","last_undegraded":"2024-09-16T02:05:30.545737+0000","last_fullsized":"2024-09-16T02:05:30.545737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:49.532084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.543830+0000","last_change":"2024-09-16T02:03:37.051477+0000","last_active":"2024-09-16T02:05:30.543830+0000","last_peered":"2024-09-16T02:05:30.543830+0000","last_clean":"2024-09-16T02:05:30.543830+0000","last_became_active":"2024-09-16T02:03:37.048063+0000","last_became_peered":"2024-09-16T02:03:37.048063+0000","last_unstale":"2024-09-16T02:05:30.543830+0000","last_undegraded":"2024-09-16T02:05:30.543830+0000","last_fullsized":"2024-09-16T02:05:30.543830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:15.973972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'14","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:31.053974+0000","last_change":"2024-09-16T02:03:34.742935+0000","last_active":"2024-09-16T02:05:31.053974+0000","last_peered":"2024-09-16T02:05:31.053974+0000","last_clean":"2024-09-16T02:05:31.053974+0000","last_became_active":"2024-09-16T02:03:34.742611+0000","last_became_peered":"2024-09-16T02:03:34.742611+0000","last_unstale":"2024-09-16T02:05:31.053974+0000","last_undegraded":"2024-09-16T02:05:31.053974+0000","last_fullsized":"2024-09-16T02:05:31.053974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:36.695220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926955+0000","last_change":"2024-09-16T02:03:37.054926+0000","last_active":"2024-09-16T02:05:30.926955+0000","last_peered":"2024-09-16T02:05:30.926955+0000","last_clean":"2024-09-16T02:05:30.926955+0000","last_became_active":"2024-09-16T02:03:37.054483+0000","last_became_peered":"2024-09-16T02:03:37.054483+0000","last_unstale":"2024-09-16T02:05:30.926955+0000","last_undegraded":"2024-09-16T02:05:30.926955+0000","last_fullsized":"2024-09-16T02:05:30.926955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545961+0000","last_change":"2024-09-16T02:03:34.739611+0000","last_active":"2024-09-16T02:05:30.545961+0000","last_peered":"2024-09-16T02:05:30.545961+0000","last_clean":"2024-09-16T02:05:30.545961+0000","last_became_active":"2024-09-16T02:03:34.738090+0000","last_became_peered":"2024-09-16T02:03:34.738090+0000","last_unstale":"2024-09-16T02:05:30.545961+0000","last_undegraded":"2024-09-16T02:05:30.545961+0000","last_fullsized":"2024-09-16T02:05:30.545961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:47.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847671+0000","last_change":"2024-09-16T02:03:37.053020+0000","last_active":"2024-09-16T02:05:30.847671+0000","last_peered":"2024-09-16T02:05:30.847671+0000","last_clean":"2024-09-16T02:05:30.847671+0000","last_became_active":"2024-09-16T02:03:37.052365+0000","last_became_peered":"2024-09-16T02:03:37.052365+0000","last_unstale":"2024-09-16T02:05:30.847671+0000","last_undegraded":"2024-09-16T02:05:30.847671+0000","last_fullsized":"2024-09-16T02:05:30.847671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:54.965028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549356+0000","last_change":"2024-09-16T02:03:34.740726+0000","last_active":"2024-09-16T02:05:30.549356+0000","last_peered":"2024-09-16T02:05:30.549356+0000","last_clean":"2024-09-16T02:05:30.549356+0000","last_became_active":"2024-09-16T02:03:34.740402+0000","last_became_peered":"2024-09-16T02:03:34.740402+0000","last_unstale":"2024-09-16T02:05:30.549356+0000","last_undegraded":"2024-09-16T02:05:30.549356+0000","last_fullsized":"2024-09-16T02:05:30.549356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:05.761646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545210+0000","last_change":"2024-09-16T02:03:37.044862+0000","last_active":"2024-09-16T02:05:30.545210+0000","last_peered":"2024-09-16T02:05:30.545210+0000","last_clean":"2024-09-16T02:05:30.545210+0000","last_became_active":"2024-09-16T02:03:37.044406+0000","last_became_peered":"2024-09-16T02:03:37.044406+0000","last_unstale":"2024-09-16T02:05:30.545210+0000","last_undegraded":"2024-09-16T02:05:30.545210+0000","last_fullsized":"2024-09-16T02:05:30.545210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:03.326380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926544+0000","last_change":"2024-09-16T02:03:34.745505+0000","last_active":"2024-09-16T02:05:30.926544+0000","last_peered":"2024-09-16T02:05:30.926544+0000","last_clean":"2024-09-16T02:05:30.926544+0000","last_became_active":"2024-09-16T02:03:34.745151+0000","last_became_peered":"2024-09-16T02:03:34.745151+0000","last_unstale":"2024-09-16T02:05:30.926544+0000","last_undegraded":"2024-09-16T02:05:30.926544+0000","last_fullsized":"2024-09-16T02:05:30.926544+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:57.401176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.547672+0000","last_change":"2024-09-16T02:03:37.046692+0000","last_active":"2024-09-16T02:05:30.547672+0000","last_peered":"2024-09-16T02:05:30.547672+0000","last_clean":"2024-09-16T02:05:30.547672+0000","last_became_active":"2024-09-16T02:03:37.046302+0000","last_became_peered":"2024-09-16T02:03:37.046302+0000","last_unstale":"2024-09-16T02:05:30.547672+0000","last_undegraded":"2024-09-16T02:05:30.547672+0000","last_fullsized":"2024-09-16T02:05:30.547672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:47.727431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.544513+0000","last_change":"2024-09-16T02:03:34.739847+0000","last_active":"2024-09-16T02:05:30.544513+0000","last_peered":"2024-09-16T02:05:30.544513+0000","last_clean":"2024-09-16T02:05:30.544513+0000","last_became_active":"2024-09-16T02:03:34.739514+0000","last_became_peered":"2024-09-16T02:03:34.739514+0000","last_unstale":"2024-09-16T02:05:30.544513+0000","last_undegraded":"2024-09-16T02:05:30.544513+0000","last_fullsized":"2024-09-16T02:05:30.544513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:44.617127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926659+0000","last_change":"2024-09-16T02:03:37.054910+0000","last_active":"2024-09-16T02:05:30.926659+0000","last_peered":"2024-09-16T02:05:30.926659+0000","last_clean":"2024-09-16T02:05:30.926659+0000","last_became_active":"2024-09-16T02:03:37.053852+0000","last_became_peered":"2024-09-16T02:03:37.053852+0000","last_unstale":"2024-09-16T02:05:30.926659+0000","last_undegraded":"2024-09-16T02:05:30.926659+0000","last_fullsized":"2024-09-16T02:05:30.926659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:29.042154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545928+0000","last_change":"2024-09-16T02:03:34.743027+0000","last_active":"2024-09-16T02:05:30.545928+0000","last_peered":"2024-09-16T02:05:30.545928+0000","last_clean":"2024-09-16T02:05:30.545928+0000","last_became_active":"2024-09-16T02:03:34.742658+0000","last_became_peered":"2024-09-16T02:03:34.742658+0000","last_unstale":"2024-09-16T02:05:30.545928+0000","last_undegraded":"2024-09-16T02:05:30.545928+0000","last_fullsized":"2024-09-16T02:05:30.545928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:36.779568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.659557+0000","last_change":"2024-09-16T02:03:37.052783+0000","last_active":"2024-09-16T02:05:30.659557+0000","last_peered":"2024-09-16T02:05:30.659557+0000","last_clean":"2024-09-16T02:05:30.659557+0000","last_became_active":"2024-09-16T02:03:37.052409+0000","last_became_peered":"2024-09-16T02:03:37.052409+0000","last_unstale":"2024-09-16T02:05:30.659557+0000","last_undegraded":"2024-09-16T02:05:30.659557+0000","last_fullsized":"2024-09-16T02:05:30.659557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:52.667379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.551560+0000","last_change":"2024-09-16T02:05:30.551560+0000","last_active":"2024-09-16T02:05:30.551560+0000","last_peered":"2024-09-16T02:05:30.551560+0000","last_clean":"2024-09-16T02:05:30.551560+0000","last_became_active":"2024-09-16T02:05:30.551063+0000","last_became_peered":"2024-09-16T02:05:30.551063+0000","last_unstale":"2024-09-16T02:05:30.551560+0000","last_undegraded":"2024-09-16T02:05:30.551560+0000","last_fullsized":"2024-09-16T02:05:30.551560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:39.081149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545662+0000","last_change":"2024-09-16T02:03:34.744240+0000","last_active":"2024-09-16T02:05:30.545662+0000","last_peered":"2024-09-16T02:05:30.545662+0000","last_clean":"2024-09-16T02:05:30.545662+0000","last_became_active":"2024-09-16T02:03:34.743898+0000","last_became_peered":"2024-09-16T02:03:34.743898+0000","last_unstale":"2024-09-16T02:05:30.545662+0000","last_undegraded":"2024-09-16T02:05:30.545662+0000","last_fullsized":"2024-09-16T02:05:30.545662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:37.820529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.549458+0000","last_change":"2024-09-16T02:03:37.055769+0000","last_active":"2024-09-16T02:05:30.549458+0000","last_peered":"2024-09-16T02:05:30.549458+0000","last_clean":"2024-09-16T02:05:30.549458+0000","last_became_active":"2024-09-16T02:03:37.054458+0000","last_became_peered":"2024-09-16T02:03:37.054458+0000","last_unstale":"2024-09-16T02:05:30.549458+0000","last_undegraded":"2024-09-16T02:05:30.549458+0000","last_fullsized":"2024-09-16T02:05:30.549458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:30.218712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.545637+0000","last_change":"2024-09-16T02:04:29.647534+0000","last_active":"2024-09-16T02:05:30.545637+0000","last_peered":"2024-09-16T02:05:30.545637+0000","last_clean":"2024-09-16T02:05:30.545637+0000","last_became_active":"2024-09-16T02:04:29.640317+0000","last_became_peered":"2024-09-16T02:04:29.640317+0000","last_unstale":"2024-09-16T02:05:30.545637+0000","last_undegraded":"2024-09-16T02:05:30.545637+0000","last_fullsized":"2024-09-16T02:05:30.545637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:14:33.490508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,6],"acting":[8,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.847438+0000","last_change":"2024-09-16T02:03:37.054370+0000","last_active":"2024-09-16T02:05:30.847438+0000","last_peered":"2024-09-16T02:05:30.847438+0000","last_clean":"2024-09-16T02:05:30.847438+0000","last_became_active":"2024-09-16T02:03:37.054051+0000","last_became_peered":"2024-09-16T02:03:37.054051+0000","last_unstale":"2024-09-16T02:05:30.847438+0000","last_undegraded":"2024-09-16T02:05:30.847438+0000","last_fullsized":"2024-09-16T02:05:30.847438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:21:42.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.927087+0000","last_change":"2024-09-16T02:03:34.739696+0000","last_active":"2024-09-16T02:05:30.927087+0000","last_peered":"2024-09-16T02:05:30.927087+0000","last_clean":"2024-09-16T02:05:30.927087+0000","last_became_active":"2024-09-16T02:03:34.738867+0000","last_became_peered":"2024-09-16T02:03:34.738867+0000","last_unstale":"2024-09-16T02:05:30.927087+0000","last_undegraded":"2024-09-16T02:05:30.927087+0000","last_fullsized":"2024-09-16T02:05:30.927087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:30.753970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.926979+0000","last_change":"2024-09-16T02:03:37.058118+0000","last_active":"2024-09-16T02:05:30.926979+0000","last_peered":"2024-09-16T02:05:30.926979+0000","last_clean":"2024-09-16T02:05:30.926979+0000","last_became_active":"2024-09-16T02:03:37.057391+0000","last_became_peered":"2024-09-16T02:03:37.057391+0000","last_unstale":"2024-09-16T02:05:30.926979+0000","last_undegraded":"2024-09-16T02:05:30.926979+0000","last_fullsized":"2024-09-16T02:05:30.926979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:36.013084+0000","last_clean_scrub_stamp":"2024-09-16T02:03:36.013084+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:39.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T02:05:30.548774+0000","last_change":"2024-09-16T02:03:34.742184+0000","last_active":"2024-09-16T02:05:30.548774+0000","last_peered":"2024-09-16T02:05:30.548774+0000","last_clean":"2024-09-16T02:05:30.548774+0000","last_became_active":"2024-09-16T02:03:34.741838+0000","last_became_peered":"2024-09-16T02:03:34.741838+0000","last_unstale":"2024-09-16T02:05:30.548774+0000","last_undegraded":"2024-09-16T02:05:30.548774+0000","last_fullsized":"2024-09-16T02:05:30.548774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T02:03:33.712486+0000","last_clean_scrub_stamp":"2024-09-16T02:03:33.712486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:57.338380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65313,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":659,"num_read_kb":404,"num_write":181,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1149,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":207864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678054,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":440447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906985,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1597440,"data_stored":1092985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037807,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":440571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266742,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":440379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495674,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712488,"statfs":{"total":95995035648,"available":95961587712,"internally_reserved":0,"allocated":1511424,"data_stored":1033802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659199,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":905216,"data_stored":439960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888131,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1519616,"data_stored":1032226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051657,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":442398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280591,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":921600,"data_stored":440271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476820,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":506571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":970752,"data_stored":500099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":925696,"data_stored":441231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T02:05:37.648 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:05:37.946 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:37 smithi172 ceph-mon[35340]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-16T02:05:37.946 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:37 smithi172 ceph-mon[35340]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-16T02:05:37.946 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:37 smithi172 ceph-mon[35340]: Cluster is now healthy 2024-09-16T02:05:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:37 smithi103 ceph-mon[30398]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-16T02:05:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:37 smithi103 ceph-mon[30398]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-16T02:05:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:37 smithi103 ceph-mon[30398]: Cluster is now healthy 2024-09-16T02:05:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:37 smithi136 ceph-mon[35323]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-16T02:05:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:37 smithi136 ceph-mon[35323]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-16T02:05:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:37 smithi136 ceph-mon[35323]: Cluster is now healthy 2024-09-16T02:05:38.051 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:38.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.529+0000 7f43ca600640 1 -- 172.21.15.103:0/377640784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c41027a0 msgr2=0x7f43c41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:38.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.529+0000 7f43ca600640 1 --2- 172.21.15.103:0/377640784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c41027a0 0x7f43c41067f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009900 tx=0x7f43ac02a410 comp rx=0 tx=0).stop 2024-09-16T02:05:38.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.529+0000 7f43ca600640 1 -- 172.21.15.103:0/377640784 shutdown_connections 2024-09-16T02:05:38.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.529+0000 7f43ca600640 1 --2- 172.21.15.103:0/377640784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43c4106f20 0x7f43c410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.529+0000 7f43ca600640 1 --2- 172.21.15.103:0/377640784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c41027a0 0x7f43c41067f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.529+0000 7f43ca600640 1 --2- 172.21.15.103:0/377640784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43c4101df0 0x7f43c41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.529+0000 7f43ca600640 1 -- 172.21.15.103:0/377640784 >> 172.21.15.103:0/377640784 conn(0x7f43c40fd8b0 msgr2=0x7f43c40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:38.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.529+0000 7f43ca600640 1 -- 172.21.15.103:0/377640784 shutdown_connections 2024-09-16T02:05:38.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.529+0000 7f43ca600640 1 -- 172.21.15.103:0/377640784 wait complete. 2024-09-16T02:05:38.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.530+0000 7f43ca600640 1 Processor -- start 2024-09-16T02:05:38.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.530+0000 7f43ca600640 1 -- start start 2024-09-16T02:05:38.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43ca600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c4101df0 0x7f43c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43ca600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43c41027a0 0x7f43c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43ca600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43c4106f20 0x7f43c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43ca600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4111fd0 con 0x7f43c4101df0 2024-09-16T02:05:38.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43ca600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4111e50 con 0x7f43c4106f20 2024-09-16T02:05:38.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43ca600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4112150 con 0x7f43c41027a0 2024-09-16T02:05:38.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43baa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43c41027a0 0x7f43c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:38.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43c4106f20 0x7f43c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:38.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43c3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c4101df0 0x7f43c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43baa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43c41027a0 0x7f43c4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:52782/0 (socket says 172.21.15.103:52782) 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43c4106f20 0x7f43c4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49762/0 (socket says 172.21.15.103:49762) 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.531+0000 7f43baa00640 1 -- 172.21.15.103:0/2791453083 learned_addr learned my addr 172.21.15.103:0/2791453083 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43c3e00640 1 -- 172.21.15.103:0/2791453083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43c41027a0 msgr2=0x7f43c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43c3e00640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43c41027a0 0x7f43c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43c3e00640 1 -- 172.21.15.103:0/2791453083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c4101df0 msgr2=0x7f43c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43c3e00640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c4101df0 0x7f43c40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43c3e00640 1 -- 172.21.15.103:0/2791453083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c41aade0 con 0x7f43c4106f20 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43baa00640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43c41027a0 0x7f43c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43c3e00640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43c4106f20 0x7f43c4074f70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00ec50 tx=0x7f43bc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:38.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43c0c00640 1 -- 172.21.15.103:0/2791453083 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc003b70 con 0x7f43c4106f20 2024-09-16T02:05:38.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43c0c00640 1 -- 172.21.15.103:0/2791453083 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc003d10 con 0x7f43c4106f20 2024-09-16T02:05:38.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43ca600640 1 -- 172.21.15.103:0/2791453083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c41aafb0 con 0x7f43c4106f20 2024-09-16T02:05:38.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43c0c00640 1 -- 172.21.15.103:0/2791453083 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc010890 con 0x7f43c4106f20 2024-09-16T02:05:38.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.532+0000 7f43ca600640 1 -- 172.21.15.103:0/2791453083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c41ab470 con 0x7f43c4106f20 2024-09-16T02:05:38.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.533+0000 7f43ca600640 1 -- 172.21.15.103:0/2791453083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c41032d0 con 0x7f43c4106f20 2024-09-16T02:05:38.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.535+0000 7f43c0c00640 1 -- 172.21.15.103:0/2791453083 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f43bc003fc0 con 0x7f43c4106f20 2024-09-16T02:05:38.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.537+0000 7f43c0c00640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43a80a3350 0x7f43a80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:38.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.537+0000 7f43c0c00640 1 -- 172.21.15.103:0/2791453083 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f43bc0d00b0 con 0x7f43c4106f20 2024-09-16T02:05:38.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.537+0000 7f43c3400640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43a80a3350 0x7f43a80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:38.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.537+0000 7f43c3400640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43a80a3350 0x7f43a80a5810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009bd0 tx=0x7f43b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:38.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.541+0000 7f43c0c00640 1 -- 172.21.15.103:0/2791453083 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43bc09cb90 con 0x7f43c4106f20 2024-09-16T02:05:38.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.753+0000 7f43ca600640 1 -- 172.21.15.103:0/2791453083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43c41021d0 con 0x7f43c4106f20 2024-09-16T02:05:38.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.754+0000 7f43c0c00640 1 -- 172.21.15.103:0/2791453083 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7f43c41021d0 con 0x7f43c4106f20 2024-09-16T02:05:38.755 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:38.755 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":24,"btime":"2024-09-16T02:05:32:550231+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:05:31.661572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24506,24505]},"id":1}]} 2024-09-16T02:05:38.755 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 24 2024-09-16T02:05:38.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 -- 172.21.15.103:0/2791453083 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43a80a3350 msgr2=0x7f43a80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:38.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43a80a3350 0x7f43a80a5810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009bd0 tx=0x7f43b4005f90 comp rx=0 tx=0).stop 2024-09-16T02:05:38.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 -- 172.21.15.103:0/2791453083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43c4106f20 msgr2=0x7f43c4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:38.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43c4106f20 0x7f43c4074f70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00ec50 tx=0x7f43bc00c6a0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 -- 172.21.15.103:0/2791453083 shutdown_connections 2024-09-16T02:05:38.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f43a80a3350 0x7f43a80a5810 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43c4106f20 0x7f43c4074f70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43c41027a0 0x7f43c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 --2- 172.21.15.103:0/2791453083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c4101df0 0x7f43c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:38.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 -- 172.21.15.103:0/2791453083 >> 172.21.15.103:0/2791453083 conn(0x7f43c40fd8b0 msgr2=0x7f43c40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:38.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 -- 172.21.15.103:0/2791453083 shutdown_connections 2024-09-16T02:05:38.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:38.756+0000 7f43ca600640 1 -- 172.21.15.103:0/2791453083 wait complete. 2024-09-16T02:05:38.770 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:38 smithi103 ceph-mon[30398]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:05:38.770 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:38 smithi103 ceph-mon[30398]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:05:38.770 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:38 smithi103 ceph-mon[30398]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:05:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:38 smithi136 ceph-mon[35323]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:05:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:38 smithi136 ceph-mon[35323]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:05:39.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:38 smithi136 ceph-mon[35323]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:05:39.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:38 smithi172 ceph-mon[35340]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:05:39.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:38 smithi172 ceph-mon[35340]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:05:39.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:38 smithi172 ceph-mon[35340]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T02:05:39.520 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:05:39.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:39 smithi103 ceph-mon[30398]: pgmap v346: 97 pgs: 97 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-16T02:05:39.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:39 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/2791453083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:39.891 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:39.924 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:39 smithi136 ceph-mon[35323]: pgmap v346: 97 pgs: 97 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-16T02:05:39.924 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:39 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/2791453083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:40.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:39 smithi172 ceph-mon[35340]: pgmap v346: 97 pgs: 97 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-16T02:05:40.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:39 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/2791453083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:05:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.403+0000 7f9a6e200640 1 -- 172.21.15.103:0/1848023117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a68104010 msgr2=0x7f9a681043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.403+0000 7f9a6e200640 1 --2- 172.21.15.103:0/1848023117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a68104010 0x7f9a681043f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009900 tx=0x7f9a5802a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.403+0000 7f9a6e200640 1 -- 172.21.15.103:0/1848023117 shutdown_connections 2024-09-16T02:05:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.403+0000 7f9a6e200640 1 --2- 172.21.15.103:0/1848023117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a68109140 0x7f9a68110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.403+0000 7f9a6e200640 1 --2- 172.21.15.103:0/1848023117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a681049c0 0x7f9a68108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.403+0000 7f9a6e200640 1 --2- 172.21.15.103:0/1848023117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a68104010 0x7f9a681043f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.403+0000 7f9a6e200640 1 -- 172.21.15.103:0/1848023117 >> 172.21.15.103:0/1848023117 conn(0x7f9a680ffa50 msgr2=0x7f9a68101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.404+0000 7f9a6e200640 1 -- 172.21.15.103:0/1848023117 shutdown_connections 2024-09-16T02:05:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.404+0000 7f9a6e200640 1 -- 172.21.15.103:0/1848023117 wait complete. 2024-09-16T02:05:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.404+0000 7f9a6e200640 1 Processor -- start 2024-09-16T02:05:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a6e200640 1 -- start start 2024-09-16T02:05:40.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a6e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a68104010 0x7f9a6819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a6e200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a681049c0 0x7f9a6819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a6e200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a68109140 0x7f9a6819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a6e200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a681141c0 con 0x7f9a68109140 2024-09-16T02:05:40.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a6e200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68114040 con 0x7f9a681049c0 2024-09-16T02:05:40.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a6e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68114340 con 0x7f9a68104010 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a67400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a68104010 0x7f9a6819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a67e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a68109140 0x7f9a6819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a67400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a68104010 0x7f9a6819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:52808/0 (socket says 172.21.15.103:52808) 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a67400640 1 -- 172.21.15.103:0/1026150636 learned_addr learned my addr 172.21.15.103:0/1026150636 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.405+0000 7f9a5ea00640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a681049c0 0x7f9a6819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.406+0000 7f9a67400640 1 -- 172.21.15.103:0/1026150636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a681049c0 msgr2=0x7f9a6819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.406+0000 7f9a67400640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a681049c0 0x7f9a6819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.406+0000 7f9a67400640 1 -- 172.21.15.103:0/1026150636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a68109140 msgr2=0x7f9a6819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.406+0000 7f9a67400640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a68109140 0x7f9a6819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.406+0000 7f9a67400640 1 -- 172.21.15.103:0/1026150636 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6819f5f0 con 0x7f9a68104010 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.406+0000 7f9a67e00640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a68109140 0x7f9a6819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:40.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.406+0000 7f9a67400640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a68104010 0x7f9a6819a640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a58004770 tx=0x7f9a58003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.406+0000 7f9a64c00640 1 -- 172.21.15.103:0/1026150636 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5803f070 con 0x7f9a68104010 2024-09-16T02:05:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.407+0000 7f9a6e200640 1 -- 172.21.15.103:0/1026150636 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6819f880 con 0x7f9a68104010 2024-09-16T02:05:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.407+0000 7f9a64c00640 1 -- 172.21.15.103:0/1026150636 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a58031ca0 con 0x7f9a68104010 2024-09-16T02:05:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.407+0000 7f9a64c00640 1 -- 172.21.15.103:0/1026150636 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58035030 con 0x7f9a68104010 2024-09-16T02:05:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.407+0000 7f9a6e200640 1 -- 172.21.15.103:0/1026150636 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a681a71c0 con 0x7f9a68104010 2024-09-16T02:05:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.408+0000 7f9a6e200640 1 -- 172.21.15.103:0/1026150636 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a681043f0 con 0x7f9a68104010 2024-09-16T02:05:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.410+0000 7f9a64c00640 1 -- 172.21.15.103:0/1026150636 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f9a58042070 con 0x7f9a68104010 2024-09-16T02:05:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.411+0000 7f9a64c00640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9a4c0a3350 0x7f9a4c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.411+0000 7f9a64c00640 1 -- 172.21.15.103:0/1026150636 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f9a580f0580 con 0x7f9a68104010 2024-09-16T02:05:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.412+0000 7f9a5ea00640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9a4c0a3350 0x7f9a4c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.412+0000 7f9a5ea00640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9a4c0a3350 0x7f9a4c0a5810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9a680ffe70 tx=0x7f9a5000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.413+0000 7f9a64c00640 1 -- 172.21.15.103:0/1026150636 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a580bcb80 con 0x7f9a68104010 2024-09-16T02:05:40.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.588+0000 7f9a6e200640 1 -- 172.21.15.103:0/1026150636 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a6819b9a0 con 0x7f9a68104010 2024-09-16T02:05:40.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.591+0000 7f9a64c00640 1 -- 172.21.15.103:0/1026150636 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23048 (secure 0 0 0) 0x7f9a580c0e80 con 0x7f9a68104010 2024-09-16T02:05:40.592 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:05:40.592 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":81,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:05:30.540044+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":10,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":7,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:05:40.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 -- 172.21.15.103:0/1026150636 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9a4c0a3350 msgr2=0x7f9a4c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9a4c0a3350 0x7f9a4c0a5810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9a680ffe70 tx=0x7f9a5000b410 comp rx=0 tx=0).stop 2024-09-16T02:05:40.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 -- 172.21.15.103:0/1026150636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a68104010 msgr2=0x7f9a6819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:40.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a68104010 0x7f9a6819a640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a58004770 tx=0x7f9a58003c10 comp rx=0 tx=0).stop 2024-09-16T02:05:40.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 -- 172.21.15.103:0/1026150636 shutdown_connections 2024-09-16T02:05:40.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9a4c0a3350 0x7f9a4c0a5810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a68109140 0x7f9a6819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a681049c0 0x7f9a6819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 --2- 172.21.15.103:0/1026150636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a68104010 0x7f9a6819a640 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:40.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 -- 172.21.15.103:0/1026150636 >> 172.21.15.103:0/1026150636 conn(0x7f9a680ffa50 msgr2=0x7f9a68100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:40.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 -- 172.21.15.103:0/1026150636 shutdown_connections 2024-09-16T02:05:40.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:40.593+0000 7f9a6e200640 1 -- 172.21.15.103:0/1026150636 wait complete. 2024-09-16T02:05:40.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:40 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1026150636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:41.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:40 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1026150636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:41.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:40 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1026150636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:05:41.264 DEBUG:tasks.fwd_scrub:Yielding 2024-09-16T02:05:41.264 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-16T02:05:41.274 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-16T02:05:41.275 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-16T02:05:41.275 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-16T02:05:41.275 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T02:05:41.275 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T02:05:41.279 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T02:05:41.335 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T02:05:41.335 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T02:05:41.335 INFO:tasks.workunit:timeout=3h 2024-09-16T02:05:41.335 INFO:tasks.workunit:cleanup=True 2024-09-16T02:05:41.336 DEBUG:teuthology.orchestra.run.smithi103:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T02:05:41.373 INFO:teuthology.orchestra.run.smithi103.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T02:05:41.373 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-16T02:05:41.373 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-16T02:05:41.373 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T02:05:41.373 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-16T02:05:41.373 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-16 02:03:57.369991933 +0000 2024-09-16T02:05:41.373 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-16 02:03:57.369991933 +0000 2024-09-16T02:05:41.373 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-16 02:05:30.800539669 +0000 2024-09-16T02:05:41.373 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-16 02:03:57.369991933 +0000 2024-09-16T02:05:41.374 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T02:05:41.374 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T02:05:41.478 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T02:05:41.517 INFO:tasks.workunit.client.0.smithi103.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T02:05:41.658 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:41.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:41 smithi103 ceph-mon[30398]: pgmap v347: 97 pgs: 97 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-16T02:05:42.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:41 smithi136 ceph-mon[35323]: pgmap v347: 97 pgs: 97 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-16T02:05:42.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:41 smithi172 ceph-mon[35340]: pgmap v347: 97 pgs: 97 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-16T02:05:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.204+0000 7efe16600640 1 -- 172.21.15.103:0/3904929842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe101049c0 msgr2=0x7efe10108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.204+0000 7efe16600640 1 --2- 172.21.15.103:0/3904929842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe101049c0 0x7efe10108a10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009900 tx=0x7efdfc02a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.204+0000 7efe16600640 1 -- 172.21.15.103:0/3904929842 shutdown_connections 2024-09-16T02:05:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.204+0000 7efe16600640 1 --2- 172.21.15.103:0/3904929842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe10109140 0x7efe10110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.204+0000 7efe16600640 1 --2- 172.21.15.103:0/3904929842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe101049c0 0x7efe10108a10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.204+0000 7efe16600640 1 --2- 172.21.15.103:0/3904929842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe10104010 0x7efe101043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.204+0000 7efe16600640 1 -- 172.21.15.103:0/3904929842 >> 172.21.15.103:0/3904929842 conn(0x7efe100ffa50 msgr2=0x7efe10101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.204+0000 7efe16600640 1 -- 172.21.15.103:0/3904929842 shutdown_connections 2024-09-16T02:05:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.205+0000 7efe16600640 1 -- 172.21.15.103:0/3904929842 wait complete. 2024-09-16T02:05:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.206+0000 7efe16600640 1 Processor -- start 2024-09-16T02:05:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.206+0000 7efe16600640 1 -- start start 2024-09-16T02:05:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe16600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe10104010 0x7efe1019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe16600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe101049c0 0x7efe1019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe16600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe10109140 0x7efe1019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe16600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe101141b0 con 0x7efe101049c0 2024-09-16T02:05:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe16600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe10114030 con 0x7efe10104010 2024-09-16T02:05:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe16600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe10114330 con 0x7efe10109140 2024-09-16T02:05:42.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe0f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe10104010 0x7efe1019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe0f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe10104010 0x7efe1019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49786/0 (socket says 172.21.15.103:49786) 2024-09-16T02:05:42.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe0ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe101049c0 0x7efe1019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe0f400640 1 -- 172.21.15.103:0/2462292358 learned_addr learned my addr 172.21.15.103:0/2462292358 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:42.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe0fe00640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe10109140 0x7efe1019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe0f400640 1 -- 172.21.15.103:0/2462292358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe10109140 msgr2=0x7efe1019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe0f400640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe10109140 0x7efe1019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe0f400640 1 -- 172.21.15.103:0/2462292358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe101049c0 msgr2=0x7efe1019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe0f400640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe101049c0 0x7efe1019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.207+0000 7efe0f400640 1 -- 172.21.15.103:0/2462292358 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1019f660 con 0x7efe10104010 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.208+0000 7efe0ea00640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe101049c0 0x7efe1019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.208+0000 7efe0f400640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe10104010 0x7efe1019a6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efe0000e9f0 tx=0x7efe0000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.208+0000 7efe07e00640 1 -- 172.21.15.103:0/2462292358 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0000cd80 con 0x7efe10104010 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.208+0000 7efe16600640 1 -- 172.21.15.103:0/2462292358 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1019f950 con 0x7efe10104010 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.208+0000 7efe07e00640 1 -- 172.21.15.103:0/2462292358 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe00005500 con 0x7efe10104010 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.208+0000 7efe07e00640 1 -- 172.21.15.103:0/2462292358 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00010690 con 0x7efe10104010 2024-09-16T02:05:42.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.208+0000 7efe16600640 1 -- 172.21.15.103:0/2462292358 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe101a7230 con 0x7efe10104010 2024-09-16T02:05:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.212+0000 7efe1905f740 1 Processor -- start 2024-09-16T02:05:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.212+0000 7efe1905f740 1 -- start start 2024-09-16T02:05:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe1905f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548939ac0 0x556548939ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe1905f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55654893a3e0 0x556548944760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe1905f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548944ca0 0x556548947090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe1905f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5565488a0290 con 0x556548939ac0 2024-09-16T02:05:42.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe1905f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556548847b40 con 0x55654893a3e0 2024-09-16T02:05:42.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe1905f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55654889c250 con 0x556548944ca0 2024-09-16T02:05:42.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe07e00640 1 -- 172.21.15.103:0/2462292358 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7efe000046c0 con 0x7efe10104010 2024-09-16T02:05:42.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe0ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548944ca0 0x556548947090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe0ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548944ca0 0x556548947090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:52830/0 (socket says 172.21.15.103:52830) 2024-09-16T02:05:42.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe0ea00640 1 -- 172.21.15.103:0/2497729374 learned_addr learned my addr 172.21.15.103:0/2497729374 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:42.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.213+0000 7efe0fe00640 1 --2- 172.21.15.103:0/2497729374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548939ac0 0x556548939ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.214+0000 7efe0f400640 1 --2- 172.21.15.103:0/2497729374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55654893a3e0 0x556548944760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.214+0000 7efe0ea00640 1 -- 172.21.15.103:0/2497729374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55654893a3e0 msgr2=0x556548944760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:42.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.214+0000 7efe0ea00640 1 --2- 172.21.15.103:0/2497729374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55654893a3e0 0x556548944760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.214+0000 7efe0ea00640 1 -- 172.21.15.103:0/2497729374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548939ac0 msgr2=0x556548939ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:42.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.214+0000 7efe0ea00640 1 --2- 172.21.15.103:0/2497729374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548939ac0 0x556548939ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.214+0000 7efe0ea00640 1 -- 172.21.15.103:0/2497729374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565486116a0 con 0x556548944ca0 2024-09-16T02:05:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.214+0000 7efe0fe00640 1 --2- 172.21.15.103:0/2497729374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548939ac0 0x556548939ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.214+0000 7efe0ea00640 1 --2- 172.21.15.103:0/2497729374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548944ca0 0x556548947090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009230 tx=0x7efdfc0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f638482bfb21c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.214+0000 7efe04c00640 1 -- 172.21.15.103:0/2497729374 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc035030 con 0x556548944ca0 2024-09-16T02:05:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.215+0000 7efe04c00640 1 -- 172.21.15.103:0/2497729374 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdfc005520 con 0x556548944ca0 2024-09-16T02:05:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.215+0000 7efe04c00640 1 -- 172.21.15.103:0/2497729374 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0035f0 con 0x556548944ca0 2024-09-16T02:05:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.215+0000 7efe07e00640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efdf40a3350 0x7efdf40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.215+0000 7efe07e00640 1 -- 172.21.15.103:0/2462292358 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7efe000d0010 con 0x7efe10104010 2024-09-16T02:05:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.215+0000 7efe0fe00640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efdf40a3350 0x7efdf40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.215+0000 7efe1905f740 1 -- 172.21.15.103:0/2497729374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548944ca0 msgr2=0x556548947090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.215+0000 7efe1905f740 1 --2- 172.21.15.103:0/2497729374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548944ca0 0x556548947090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009230 tx=0x7efdfc0040c0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.216+0000 7efe1905f740 1 -- 172.21.15.103:0/2497729374 shutdown_connections 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.216+0000 7efe1905f740 1 --2- 172.21.15.103:0/2497729374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548944ca0 0x556548947090 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.216+0000 7efe1905f740 1 --2- 172.21.15.103:0/2497729374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55654893a3e0 0x556548944760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.216+0000 7efe1905f740 1 --2- 172.21.15.103:0/2497729374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548939ac0 0x556548939ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.216+0000 7efe1905f740 1 -- 172.21.15.103:0/2497729374 >> 172.21.15.103:0/2497729374 conn(0x556548936050 msgr2=0x556548936430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.216+0000 7efe1905f740 1 -- 172.21.15.103:0/2497729374 shutdown_connections 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.216+0000 7efe1905f740 1 -- 172.21.15.103:0/2497729374 wait complete. 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.216+0000 7efe0fe00640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efdf40a3350 0x7efdf40a5810 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe08005fd0 tx=0x7efe080096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.216+0000 7efe1905f740 1 -- start start 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.217+0000 7efe1905f740 1 Processor -- start 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.217+0000 7efe1905f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548939ac0 0x5565489445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.217+0000 7efe1905f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55654893a3e0 0x5565489418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.217+0000 7efe1905f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548944ca0 0x556548941e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.217+0000 7efe1905f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556548947910 con 0x556548944ca0 2024-09-16T02:05:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.217+0000 7efe1905f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55654885afa0 con 0x556548939ac0 2024-09-16T02:05:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.217+0000 7efe1905f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556548947a90 con 0x55654893a3e0 2024-09-16T02:05:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.217+0000 7efe1905f740 10 client.0 _ll_register_callbacks cb 0x5565485aeba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548944ca0 0x556548941e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548939ac0 0x5565489445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548939ac0 0x5565489445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49808/0 (socket says 172.21.15.103:49808) 2024-09-16T02:05:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0f400640 1 -- 172.21.15.103:0/2427628650 learned_addr learned my addr 172.21.15.103:0/2427628650 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548944ca0 0x556548941e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48180/0 (socket says 172.21.15.103:48180) 2024-09-16T02:05:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0ea00640 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55654893a3e0 0x5565489418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0f400640 1 -- 172.21.15.103:0/2427628650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55654893a3e0 msgr2=0x5565489418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0f400640 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55654893a3e0 0x5565489418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0f400640 1 -- 172.21.15.103:0/2427628650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548944ca0 msgr2=0x556548941e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0f400640 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548944ca0 0x556548941e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0f400640 1 -- 172.21.15.103:0/2427628650 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556548942630 con 0x556548939ac0 2024-09-16T02:05:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efe0f400640 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548939ac0 0x5565489445f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe0009c0f0 tx=0x7efe00098750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24190272bb7bf4ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.218+0000 7efdf9600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:05:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efdf9600640 1 -- 172.21.15.103:0/2427628650 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe000a4dd0 con 0x556548939ac0 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efdf9600640 1 -- 172.21.15.103:0/2427628650 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7efe0009b840 con 0x556548939ac0 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efe1905f740 1 -- 172.21.15.103:0/2427628650 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc005520 con 0x556548939ac0 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efdf9600640 1 -- 172.21.15.103:0/2427628650 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efe000b5020 con 0x556548939ac0 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efe1905f740 10 client.25085 fetch_fsmap learned FSMap version 24 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efe1905f740 1 -- 172.21.15.103:0/2427628650 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556548953900 con 0x556548939ac0 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efdf9600640 1 -- 172.21.15.103:0/2427628650 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdfc005520 con 0x556548939ac0 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efdf9600640 1 -- 172.21.15.103:0/2427628650 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7efe000b51c0 con 0x556548939ac0 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efe1905f740 10 client.25085 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efe1905f740 10 client.25085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.219+0000 7efe1905f740 20 client.25085 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.220+0000 7efe1905f740 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556548954dc0 0x5565489571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.220+0000 7efe1905f740 4 client.25085 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.220+0000 7efe1905f740 1 -- 172.21.15.103:0/2427628650 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565488ad800 con 0x556548954dc0 2024-09-16T02:05:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.220+0000 7efe0ea00640 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556548954dc0 0x5565489571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.220+0000 7efe0ea00640 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556548954dc0 0x5565489571b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84522179c6e12ce4 server_cookie=b9bd2beefe2ccbcc in_seq=0 out_seq=0 2024-09-16T02:05:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.220+0000 7efdf9600640 10 client.25085 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.221+0000 7efdf9600640 1 -- 172.21.15.103:0/2427628650 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565488ad800 con 0x556548954dc0 2024-09-16T02:05:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.221+0000 7efdf9600640 10 client.25085 handle_command_reply: tid=0 2024-09-16T02:05:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.221+0000 7efe1905f740 1 client.25085 shutdown 2024-09-16T02:05:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 1 -- 172.21.15.103:0/2427628650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548939ac0 msgr2=0x5565489445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548939ac0 0x5565489445f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe0009c0f0 tx=0x7efe00098750 comp rx=0 tx=0).stop 2024-09-16T02:05:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 1 -- 172.21.15.103:0/2427628650 shutdown_connections 2024-09-16T02:05:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556548954dc0 0x5565489571b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556548944ca0 0x556548941e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55654893a3e0 0x5565489418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 1 --2- 172.21.15.103:0/2427628650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548939ac0 0x5565489445f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 1 -- 172.21.15.103:0/2427628650 >> 172.21.15.103:0/2427628650 conn(0x556548936050 msgr2=0x55654893bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efdf9600640 0 client.25085 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 1 -- 172.21.15.103:0/2427628650 shutdown_connections 2024-09-16T02:05:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 1 -- 172.21.15.103:0/2427628650 wait complete. 2024-09-16T02:05:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.222+0000 7efe1905f740 20 client.25085 trim_cache size 0 max 16384 2024-09-16T02:05:42.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.247+0000 7efe1905f740 1 Processor -- start 2024-09-16T02:05:42.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.247+0000 7efe1905f740 1 -- start start 2024-09-16T02:05:42.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.247+0000 7efe1905f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 0x55654893ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.247+0000 7efe1905f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x55654895ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.247+0000 7efe1905f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 0x55654895cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.247+0000 7efe1905f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe101b21f0 con 0x55654893a3e0 2024-09-16T02:05:42.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.247+0000 7efe1905f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe101afac0 con 0x556548944ca0 2024-09-16T02:05:42.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.247+0000 7efe1905f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe101ad910 con 0x556548939ac0 2024-09-16T02:05:42.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x55654895ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x55654895ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48190/0 (socket says 172.21.15.103:48190) 2024-09-16T02:05:42.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0ea00640 1 -- 172.21.15.103:0/553190474 learned_addr learned my addr 172.21.15.103:0/553190474 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:42.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0f400640 1 --2- 172.21.15.103:0/553190474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 0x55654893ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0fe00640 1 --2- 172.21.15.103:0/553190474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 0x55654895cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0ea00640 1 -- 172.21.15.103:0/553190474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 msgr2=0x55654893ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0ea00640 1 --2- 172.21.15.103:0/553190474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 0x55654893ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0ea00640 1 -- 172.21.15.103:0/553190474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 msgr2=0x55654895cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0ea00640 1 --2- 172.21.15.103:0/553190474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 0x55654895cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0ea00640 1 -- 172.21.15.103:0/553190474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556548947910 con 0x55654893a3e0 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.248+0000 7efe0fe00640 1 --2- 172.21.15.103:0/553190474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 0x55654895cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.249+0000 7efe0ea00640 1 --2- 172.21.15.103:0/553190474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x55654895ca30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009230 tx=0x7efdfc005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2295a8fe94bdbbd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.249+0000 7efdfbe00640 1 -- 172.21.15.103:0/553190474 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc035040 con 0x55654893a3e0 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.249+0000 7efdfbe00640 1 -- 172.21.15.103:0/553190474 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdfc031d50 con 0x55654893a3e0 2024-09-16T02:05:42.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.249+0000 7efdfbe00640 1 -- 172.21.15.103:0/553190474 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc04e440 con 0x55654893a3e0 2024-09-16T02:05:42.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.250+0000 7efe1905f740 1 -- 172.21.15.103:0/553190474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 msgr2=0x55654895ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.250+0000 7efe1905f740 1 --2- 172.21.15.103:0/553190474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x55654895ca30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009230 tx=0x7efdfc005b90 comp rx=0 tx=0).stop 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.250+0000 7efe1905f740 1 -- 172.21.15.103:0/553190474 shutdown_connections 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.250+0000 7efe1905f740 1 --2- 172.21.15.103:0/553190474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 0x55654895cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.250+0000 7efe1905f740 1 --2- 172.21.15.103:0/553190474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x55654895ca30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.250+0000 7efe1905f740 1 --2- 172.21.15.103:0/553190474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 0x55654893ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.250+0000 7efe1905f740 1 -- 172.21.15.103:0/553190474 >> 172.21.15.103:0/553190474 conn(0x556548936050 msgr2=0x55654893c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.250+0000 7efe1905f740 1 -- 172.21.15.103:0/553190474 shutdown_connections 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.250+0000 7efe1905f740 1 -- 172.21.15.103:0/553190474 wait complete. 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.250+0000 7efe1905f740 1 -- start start 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.251+0000 7efe1905f740 1 Processor -- start 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.251+0000 7efe1905f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 0x556548948b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.251+0000 7efe1905f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x556548949070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.251+0000 7efe1905f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 0x5565489495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.251+0000 7efe1905f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe101adf80 con 0x55654893a3e0 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.251+0000 7efe1905f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe101af3f0 con 0x556548944ca0 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.251+0000 7efe1905f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe101ae650 con 0x556548939ac0 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.251+0000 7efe1905f740 10 client.0 _ll_register_callbacks cb 0x556548942800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x556548949070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 0x5565489495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x556548949070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48204/0 (socket says 172.21.15.103:48204) 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0ea00640 1 -- 172.21.15.103:0/1687793148 learned_addr learned my addr 172.21.15.103:0/1687793148 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0f400640 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 0x556548948b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0f400640 1 -- 172.21.15.103:0/1687793148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 msgr2=0x5565489495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0f400640 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 0x5565489495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0f400640 1 -- 172.21.15.103:0/1687793148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 msgr2=0x556548949070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:42.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0f400640 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x556548949070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0f400640 1 -- 172.21.15.103:0/1687793148 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565489beb00 con 0x556548939ac0 2024-09-16T02:05:42.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efe0f400640 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 0x556548948b30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efe0009c380 tx=0x7efe0009c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=326bb2ebafb2bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:42.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.252+0000 7efdf9600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:05:42.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.253+0000 7efdf9600640 1 -- 172.21.15.103:0/1687793148 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0009bf30 con 0x556548939ac0 2024-09-16T02:05:42.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.253+0000 7efdf9600640 1 -- 172.21.15.103:0/1687793148 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7efe000a4bd0 con 0x556548939ac0 2024-09-16T02:05:42.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.253+0000 7efe1905f740 1 -- 172.21.15.103:0/1687793148 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc031d50 con 0x556548939ac0 2024-09-16T02:05:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.253+0000 7efdf9600640 1 -- 172.21.15.103:0/1687793148 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efe0009caf0 con 0x556548939ac0 2024-09-16T02:05:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.253+0000 7efe1905f740 10 client.25075 fetch_fsmap learned FSMap version 24 2024-09-16T02:05:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.253+0000 7efe1905f740 1 -- 172.21.15.103:0/1687793148 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5565489bfe20 con 0x556548939ac0 2024-09-16T02:05:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.253+0000 7efdf9600640 1 -- 172.21.15.103:0/1687793148 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdfc031d50 con 0x556548939ac0 2024-09-16T02:05:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.254+0000 7efdf9600640 1 -- 172.21.15.103:0/1687793148 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7efe000d30e0 con 0x556548939ac0 2024-09-16T02:05:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.254+0000 7efe1905f740 10 client.25075 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:05:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.254+0000 7efe1905f740 10 client.25075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.254+0000 7efe1905f740 20 client.25075 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.254+0000 7efe1905f740 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5565489c0490 0x5565489c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.254+0000 7efe1905f740 4 client.25075 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.254+0000 7efe1905f740 1 -- 172.21.15.103:0/1687793148 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5565488a7d10 con 0x5565489c0490 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.254+0000 7efe0ea00640 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5565489c0490 0x5565489c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.255+0000 7efe0ea00640 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5565489c0490 0x5565489c2880 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62946dc2733c739 server_cookie=64f2aa70db5e85cf in_seq=0 out_seq=0 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.255+0000 7efdf9600640 10 client.25075 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efdf9600640 1 -- 172.21.15.103:0/1687793148 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5565488a7d10 con 0x5565489c0490 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efdf9600640 10 client.25075 handle_command_reply: tid=0 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efe1905f740 1 client.25075 shutdown 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efe1905f740 1 -- 172.21.15.103:0/1687793148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 msgr2=0x556548948b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efe1905f740 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 0x556548948b30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efe0009c380 tx=0x7efe0009c280 comp rx=0 tx=0).stop 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efe1905f740 1 -- 172.21.15.103:0/1687793148 shutdown_connections 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efe1905f740 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5565489c0490 0x5565489c2880 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efe1905f740 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556548944ca0 0x5565489495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efe1905f740 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55654893a3e0 0x556548949070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efe1905f740 1 --2- 172.21.15.103:0/1687793148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556548939ac0 0x556548948b30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efdf9600640 0 client.25075 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:42.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.256+0000 7efe1905f740 1 -- 172.21.15.103:0/1687793148 >> 172.21.15.103:0/1687793148 conn(0x556548936050 msgr2=0x5565489351c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:42.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.257+0000 7efe1905f740 1 -- 172.21.15.103:0/1687793148 shutdown_connections 2024-09-16T02:05:42.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.257+0000 7efe1905f740 1 -- 172.21.15.103:0/1687793148 wait complete. 2024-09-16T02:05:42.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.257+0000 7efe1905f740 20 client.25075 trim_cache size 0 max 16384 2024-09-16T02:05:42.260 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:05:42.260 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-16T02:05:42.260 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "00b0dec2-af4e-4126-9d72-c793f70107ae", 2024-09-16T02:05:42.261 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-16T02:05:42.261 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:05:42.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.257+0000 7efe16600640 1 -- 172.21.15.103:0/2462292358 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efdf40a3350 msgr2=0x7efdf40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.257+0000 7efe16600640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efdf40a3350 0x7efdf40a5810 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe08005fd0 tx=0x7efe080096a0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.258+0000 7efe16600640 1 -- 172.21.15.103:0/2462292358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe10104010 msgr2=0x7efe1019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:42.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.258+0000 7efe16600640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe10104010 0x7efe1019a6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efe0000e9f0 tx=0x7efe0000edf0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.258+0000 7efe16600640 1 -- 172.21.15.103:0/2462292358 shutdown_connections 2024-09-16T02:05:42.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.258+0000 7efe16600640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efdf40a3350 0x7efdf40a5810 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.258+0000 7efe16600640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe10109140 0x7efe1019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.258+0000 7efe16600640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe101049c0 0x7efe1019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.258+0000 7efe16600640 1 --2- 172.21.15.103:0/2462292358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe10104010 0x7efe1019a6b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:42.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.258+0000 7efe16600640 1 -- 172.21.15.103:0/2462292358 >> 172.21.15.103:0/2462292358 conn(0x7efe100ffa50 msgr2=0x7efe10100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:42.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.258+0000 7efe16600640 1 -- 172.21.15.103:0/2462292358 shutdown_connections 2024-09-16T02:05:42.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:42.258+0000 7efe16600640 1 -- 172.21.15.103:0/2462292358 wait complete. 2024-09-16T02:05:42.952 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:42 smithi103 ceph-mon[30398]: scrub queued for path: / 2024-09-16T02:05:42.952 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:42 smithi103 ceph-mon[30398]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:42.952 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:42 smithi103 ceph-mon[30398]: scrub summary: active paths [/] 2024-09-16T02:05:42.952 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:42 smithi103 ceph-mon[30398]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:42 smithi136 ceph-mon[35323]: scrub queued for path: / 2024-09-16T02:05:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:42 smithi136 ceph-mon[35323]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:42 smithi136 ceph-mon[35323]: scrub summary: active paths [/] 2024-09-16T02:05:43.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:42 smithi136 ceph-mon[35323]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:43.026 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:05:43.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:42 smithi172 ceph-mon[35340]: scrub queued for path: / 2024-09-16T02:05:43.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:42 smithi172 ceph-mon[35340]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:43.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:42 smithi172 ceph-mon[35340]: scrub summary: active paths [/] 2024-09-16T02:05:43.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:42 smithi172 ceph-mon[35340]: scrub summary: idle+waiting paths [/] 2024-09-16T02:05:43.453 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:05:43.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:43 smithi103 ceph-mon[30398]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-16T02:05:44.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:43 smithi136 ceph-mon[35323]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-16T02:05:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.011+0000 7f9dd0400640 1 -- 172.21.15.103:0/459434746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc8101de0 msgr2=0x7f9dc81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.011+0000 7f9dd0400640 1 --2- 172.21.15.103:0/459434746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc8101de0 0x7f9dc81021c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009900 tx=0x7f9dbc02a3f0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.012+0000 7f9dd0400640 1 -- 172.21.15.103:0/459434746 shutdown_connections 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.012+0000 7f9dd0400640 1 --2- 172.21.15.103:0/459434746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc8106f10 0x7f9dc810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.012+0000 7f9dd0400640 1 --2- 172.21.15.103:0/459434746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc8102790 0x7f9dc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.012+0000 7f9dd0400640 1 --2- 172.21.15.103:0/459434746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc8101de0 0x7f9dc81021c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.012+0000 7f9dd0400640 1 -- 172.21.15.103:0/459434746 >> 172.21.15.103:0/459434746 conn(0x7f9dc80fd8c0 msgr2=0x7f9dc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.012+0000 7f9dd0400640 1 -- 172.21.15.103:0/459434746 shutdown_connections 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.012+0000 7f9dd0400640 1 -- 172.21.15.103:0/459434746 wait complete. 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dd0400640 1 Processor -- start 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dd0400640 1 -- start start 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dd0400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc8101de0 0x7f9dc819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dd0400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc8102790 0x7f9dc819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dd0400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc8106f10 0x7f9dc819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dd0400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc8111fc0 con 0x7f9dc8101de0 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dd0400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc8111e40 con 0x7f9dc8102790 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dd0400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc8112140 con 0x7f9dc8106f10 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dcda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc8101de0 0x7f9dc819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dcda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc8101de0 0x7f9dc819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48222/0 (socket says 172.21.15.103:48222) 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dcda00640 1 -- 172.21.15.103:0/150129060 learned_addr learned my addr 172.21.15.103:0/150129060 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dcd000640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc8102790 0x7f9dc819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.013+0000 7f9dce400640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc8106f10 0x7f9dc819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.014+0000 7f9dcda00640 1 -- 172.21.15.103:0/150129060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc8106f10 msgr2=0x7f9dc819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.014+0000 7f9dcda00640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc8106f10 0x7f9dc819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.014+0000 7f9dcda00640 1 -- 172.21.15.103:0/150129060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc8102790 msgr2=0x7f9dc819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.014+0000 7f9dcda00640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc8102790 0x7f9dc819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.014+0000 7f9dcda00640 1 -- 172.21.15.103:0/150129060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc819f690 con 0x7f9dc8101de0 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.014+0000 7f9dcda00640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc8101de0 0x7f9dc819a6e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc004770 tx=0x7f9dbc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.014+0000 7f9dc6000640 1 -- 172.21.15.103:0/150129060 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc03f070 con 0x7f9dc8101de0 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.014+0000 7f9dc6000640 1 -- 172.21.15.103:0/150129060 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dbc031ca0 con 0x7f9dc8101de0 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.014+0000 7f9dd0400640 1 -- 172.21.15.103:0/150129060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc819f920 con 0x7f9dc8101de0 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.015+0000 7f9dc6000640 1 -- 172.21.15.103:0/150129060 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc035030 con 0x7f9dc8101de0 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.015+0000 7f9dd0400640 1 -- 172.21.15.103:0/150129060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc81a7260 con 0x7f9dc8101de0 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.018+0000 7f9dc6000640 1 -- 172.21.15.103:0/150129060 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f9dbc042070 con 0x7f9dc8101de0 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.018+0000 7f9dd3009740 1 Processor -- start 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.018+0000 7f9dd3009740 1 -- start start 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.018+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded8ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.018+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594ded9a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.018+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 0x5594ded9d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.018+0000 7f9dd3009740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5594decf63e0 con 0x5594ded904b0 2024-09-16T02:05:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.018+0000 7f9dd3009740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5594dec9dd80 con 0x5594ded8fb90 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.018+0000 7f9dd3009740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594decf23a0 con 0x5594ded9ad70 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594ded9a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594ded9a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48234/0 (socket says 172.21.15.103:48234) 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcda00640 1 -- 172.21.15.103:0/2536706218 learned_addr learned my addr 172.21.15.103:0/2536706218 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dce400640 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded8ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcd000640 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 0x5594ded9d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcda00640 1 -- 172.21.15.103:0/2536706218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 msgr2=0x5594ded9d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcda00640 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 0x5594ded9d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcda00640 1 -- 172.21.15.103:0/2536706218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 msgr2=0x5594ded8ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcda00640 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded8ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcda00640 1 -- 172.21.15.103:0/2536706218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594decbcae0 con 0x5594ded904b0 2024-09-16T02:05:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dce400640 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded8ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcd000640 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 0x5594ded9d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dc6000640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9db00a3350 0x7f9db00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dc6000640 1 -- 172.21.15.103:0/150129060 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f9dbc0f0220 con 0x7f9dc8101de0 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.019+0000 7f9dcda00640 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594ded9a830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0f8480 tx=0x7f9dbc0f8570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a3b088aca77e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.020+0000 7f9dce400640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9db00a3350 0x7f9db00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.020+0000 7f9dbaa00640 1 -- 172.21.15.103:0/2536706218 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc049070 con 0x5594ded904b0 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.020+0000 7f9dbaa00640 1 -- 172.21.15.103:0/2536706218 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dbc031e50 con 0x5594ded904b0 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.020+0000 7f9dbaa00640 1 -- 172.21.15.103:0/2536706218 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc0b8950 con 0x5594ded904b0 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dd3009740 1 -- 172.21.15.103:0/2536706218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 msgr2=0x5594ded9a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dce400640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9db00a3350 0x7f9db00a5810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0005fd0 tx=0x7f9dc0009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dd3009740 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594ded9a830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0f8480 tx=0x7f9dbc0f8570 comp rx=0 tx=0).stop 2024-09-16T02:05:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dd3009740 1 -- 172.21.15.103:0/2536706218 shutdown_connections 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dd3009740 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 0x5594ded9d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dd3009740 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594ded9a830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dd3009740 1 --2- 172.21.15.103:0/2536706218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded8ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dd3009740 1 -- 172.21.15.103:0/2536706218 >> 172.21.15.103:0/2536706218 conn(0x5594ded8c120 msgr2=0x5594ded8c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dd3009740 1 -- 172.21.15.103:0/2536706218 shutdown_connections 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dd3009740 1 -- 172.21.15.103:0/2536706218 wait complete. 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.021+0000 7f9dd3009740 1 -- start start 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dd3009740 1 Processor -- start 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded97c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded904b0 0x5594ded98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 0x5594ded98680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dd3009740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5594ded9d700 con 0x5594ded9ad70 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dd3009740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5594decb10f0 con 0x5594ded8fb90 2024-09-16T02:05:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dd3009740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594ded9d880 con 0x5594ded904b0 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dd3009740 10 client.0 _ll_register_callbacks cb 0x5594deca29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dcda00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded97c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dcda00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded97c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49868/0 (socket says 172.21.15.103:49868) 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dcda00640 1 -- 172.21.15.103:0/3457407826 learned_addr learned my addr 172.21.15.103:0/3457407826 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dce400640 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 0x5594ded98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.022+0000 7f9dcd000640 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded904b0 0x5594ded98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9dcda00640 1 -- 172.21.15.103:0/3457407826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded904b0 msgr2=0x5594ded98140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9dcda00640 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded904b0 0x5594ded98140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9dcda00640 1 -- 172.21.15.103:0/3457407826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 msgr2=0x5594ded98680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9dcda00640 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 0x5594ded98680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9dcda00640 1 -- 172.21.15.103:0/3457407826 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594dea673c0 con 0x5594ded8fb90 2024-09-16T02:05:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9dcda00640 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded97c00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0f4fd0 tx=0x7f9dbc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f0d308439a6427e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9daf400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9daf400640 1 -- 172.21.15.103:0/3457407826 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc049070 con 0x5594ded8fb90 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9daf400640 1 -- 172.21.15.103:0/3457407826 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f9dbc0edd40 con 0x5594ded8fb90 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9dd3009740 1 -- 172.21.15.103:0/3457407826 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dbc031e50 con 0x5594ded8fb90 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9daf400640 1 -- 172.21.15.103:0/3457407826 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9dbc0bc010 con 0x5594ded8fb90 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9dd3009740 10 client.25103 fetch_fsmap learned FSMap version 24 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.023+0000 7f9dd3009740 1 -- 172.21.15.103:0/3457407826 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5594ded9d700 con 0x5594ded8fb90 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.024+0000 7f9daf400640 1 -- 172.21.15.103:0/3457407826 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dbc031e50 con 0x5594ded8fb90 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.024+0000 7f9daf400640 1 -- 172.21.15.103:0/3457407826 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9dbc03b5f0 con 0x5594ded8fb90 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.024+0000 7f9dd3009740 10 client.25103 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.024+0000 7f9dd3009740 10 client.25103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.024+0000 7f9dd3009740 20 client.25103 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.024+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5594dedaa620 0x5594dedaca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.024+0000 7f9dd3009740 4 client.25103 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.024+0000 7f9dd3009740 1 -- 172.21.15.103:0/3457407826 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594ded03950 con 0x5594dedaa620 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.024+0000 7f9dcd000640 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5594dedaa620 0x5594dedaca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.025+0000 7f9dcd000640 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5594dedaa620 0x5594dedaca70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ee1f6932e91a9a server_cookie=dcdde38b63979bde in_seq=0 out_seq=0 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.025+0000 7f9daf400640 10 client.25103 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9daf400640 1 -- 172.21.15.103:0/3457407826 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594ded03950 con 0x5594dedaa620 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9daf400640 10 client.25103 handle_command_reply: tid=0 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 client.25103 shutdown 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 -- 172.21.15.103:0/3457407826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 msgr2=0x5594ded97c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded97c00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0f4fd0 tx=0x7f9dbc009260 comp rx=0 tx=0).stop 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 -- 172.21.15.103:0/3457407826 shutdown_connections 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5594dedaa620 0x5594dedaca70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 0x5594ded98680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded904b0 0x5594ded98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3457407826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594ded97c00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9daf400640 0 client.25103 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 -- 172.21.15.103:0/3457407826 >> 172.21.15.103:0/3457407826 conn(0x5594ded8c120 msgr2=0x5594ded91e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 -- 172.21.15.103:0/3457407826 shutdown_connections 2024-09-16T02:05:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 1 -- 172.21.15.103:0/3457407826 wait complete. 2024-09-16T02:05:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.026+0000 7f9dd3009740 20 client.25103 trim_cache size 0 max 16384 2024-09-16T02:05:44.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:43 smithi172 ceph-mon[35340]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-16T02:05:44.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.044+0000 7f9dd3009740 1 Processor -- start 2024-09-16T02:05:44.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.044+0000 7f9dd3009740 1 -- start start 2024-09-16T02:05:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.044+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594dedb2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.044+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594dedb2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.044+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 0x5594ded94e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.044+0000 7f9dd3009740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81b2300 con 0x5594ded904b0 2024-09-16T02:05:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.044+0000 7f9dd3009740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81afbd0 con 0x5594ded8fb90 2024-09-16T02:05:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.044+0000 7f9dd3009740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81ada20 con 0x5594ded9ad70 2024-09-16T02:05:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcd000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594dedb2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcd000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594dedb2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48262/0 (socket says 172.21.15.103:48262) 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcda00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594dedb2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcd000640 1 -- 172.21.15.103:0/2699592069 learned_addr learned my addr 172.21.15.103:0/2699592069 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dce400640 1 --2- 172.21.15.103:0/2699592069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 0x5594ded94e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcd000640 1 -- 172.21.15.103:0/2699592069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 msgr2=0x5594ded94e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcd000640 1 --2- 172.21.15.103:0/2699592069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 0x5594ded94e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcd000640 1 -- 172.21.15.103:0/2699592069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 msgr2=0x5594dedb2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcd000640 1 --2- 172.21.15.103:0/2699592069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594dedb2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcd000640 1 -- 172.21.15.103:0/2699592069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ded8bf50 con 0x5594ded904b0 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcda00640 1 --2- 172.21.15.103:0/2699592069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594dedb2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.045+0000 7f9dcd000640 1 --2- 172.21.15.103:0/2699592069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594dedb2880 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9db400b620 tx=0x7f9db400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff35606a15c8364 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dba000640 1 -- 172.21.15.103:0/2699592069 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4003e80 con 0x5594ded904b0 2024-09-16T02:05:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dba000640 1 -- 172.21.15.103:0/2699592069 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db4005500 con 0x5594ded904b0 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dba000640 1 -- 172.21.15.103:0/2699592069 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4011680 con 0x5594ded904b0 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dd3009740 1 -- 172.21.15.103:0/2699592069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 msgr2=0x5594dedb2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dd3009740 1 --2- 172.21.15.103:0/2699592069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594dedb2880 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9db400b620 tx=0x7f9db400bf50 comp rx=0 tx=0).stop 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dd3009740 1 -- 172.21.15.103:0/2699592069 shutdown_connections 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dd3009740 1 --2- 172.21.15.103:0/2699592069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded9ad70 0x5594ded94e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dd3009740 1 --2- 172.21.15.103:0/2699592069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded904b0 0x5594dedb2880 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dd3009740 1 --2- 172.21.15.103:0/2699592069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded8fb90 0x5594dedb2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dd3009740 1 -- 172.21.15.103:0/2699592069 >> 172.21.15.103:0/2699592069 conn(0x5594ded8c120 msgr2=0x5594ded8b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.046+0000 7f9dd3009740 1 -- 172.21.15.103:0/2699592069 shutdown_connections 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.047+0000 7f9dd3009740 1 -- 172.21.15.103:0/2699592069 wait complete. 2024-09-16T02:05:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.047+0000 7f9dd3009740 1 -- start start 2024-09-16T02:05:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.047+0000 7f9dd3009740 1 Processor -- start 2024-09-16T02:05:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.047+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded8fb90 0x5594ded9ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded904b0 0x5594ded9f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dd3009740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 0x5594dee14310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dd3009740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81ae090 con 0x5594ded9ad70 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dd3009740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81af500 con 0x5594ded904b0 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dd3009740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81ae760 con 0x5594ded8fb90 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dd3009740 10 client.0 _ll_register_callbacks cb 0x7f9dbc0f8300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dce400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 0x5594dee14310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dce400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 0x5594dee14310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48278/0 (socket says 172.21.15.103:48278) 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dcda00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded8fb90 0x5594ded9ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dce400640 1 -- 172.21.15.103:0/3774893020 learned_addr learned my addr 172.21.15.103:0/3774893020 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dcd000640 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded904b0 0x5594ded9f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dcda00640 1 -- 172.21.15.103:0/3774893020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded904b0 msgr2=0x5594ded9f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dcda00640 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded904b0 0x5594ded9f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dcda00640 1 -- 172.21.15.103:0/3774893020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 msgr2=0x5594dee14310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dcda00640 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 0x5594dee14310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dcda00640 1 -- 172.21.15.103:0/3774893020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594dee14850 con 0x5594ded8fb90 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.048+0000 7f9dce400640 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 0x5594dee14310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.049+0000 7f9dcda00640 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded8fb90 0x5594ded9ef30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0ba960 tx=0x7f9dbc0edf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=397fb5f6f908bcad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.049+0000 7f9daf400640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.049+0000 7f9daf400640 1 -- 172.21.15.103:0/3774893020 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc049070 con 0x5594ded8fb90 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.049+0000 7f9daf400640 1 -- 172.21.15.103:0/3774893020 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f9dbc0c4de0 con 0x5594ded8fb90 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.049+0000 7f9dd3009740 1 -- 172.21.15.103:0/3774893020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9db4005500 con 0x5594ded8fb90 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.049+0000 7f9daf400640 1 -- 172.21.15.103:0/3774893020 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9dbc0bc1f0 con 0x5594ded8fb90 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.049+0000 7f9dd3009740 10 client.25093 fetch_fsmap learned FSMap version 24 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.049+0000 7f9dd3009740 1 -- 172.21.15.103:0/3774893020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5594dee14850 con 0x5594ded8fb90 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.049+0000 7f9daf400640 1 -- 172.21.15.103:0/3774893020 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db4005500 con 0x5594ded8fb90 2024-09-16T02:05:44.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.050+0000 7f9daf400640 1 -- 172.21.15.103:0/3774893020 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9dbc0bc910 con 0x5594ded8fb90 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.050+0000 7f9dd3009740 10 client.25093 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.050+0000 7f9dd3009740 10 client.25093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.050+0000 7f9dd3009740 20 client.25093 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.050+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5594dee15ee0 0x5594dee182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.050+0000 7f9dd3009740 4 client.25093 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.050+0000 7f9dd3009740 1 -- 172.21.15.103:0/3774893020 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594decfde60 con 0x5594dee15ee0 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.050+0000 7f9dcd000640 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5594dee15ee0 0x5594dee182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dcd000640 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5594dee15ee0 0x5594dee182d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f90540ed2bc21eb9 server_cookie=5029d5a3946bc496 in_seq=0 out_seq=0 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9daf400640 10 client.25093 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9daf400640 1 -- 172.21.15.103:0/3774893020 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5594decfde60 con 0x5594dee15ee0 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9daf400640 10 client.25093 handle_command_reply: tid=0 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 client.25093 shutdown 2024-09-16T02:05:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 -- 172.21.15.103:0/3774893020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded8fb90 msgr2=0x5594ded9ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded8fb90 0x5594ded9ef30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0ba960 tx=0x7f9dbc0edf70 comp rx=0 tx=0).stop 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 -- 172.21.15.103:0/3774893020 shutdown_connections 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5594dee15ee0 0x5594dee182d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594ded9ad70 0x5594dee14310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594ded904b0 0x5594ded9f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 --2- 172.21.15.103:0/3774893020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594ded8fb90 0x5594ded9ef30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9daf400640 0 client.25093 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 -- 172.21.15.103:0/3774893020 >> 172.21.15.103:0/3774893020 conn(0x5594ded8c120 msgr2=0x5594deda5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 -- 172.21.15.103:0/3774893020 shutdown_connections 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 1 -- 172.21.15.103:0/3774893020 wait complete. 2024-09-16T02:05:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.051+0000 7f9dd3009740 20 client.25093 trim_cache size 0 max 16384 2024-09-16T02:05:44.057 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:05:44.057 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T02:05:44.057 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-16T02:05:44.057 INFO:teuthology.orchestra.run.smithi103.stdout: "00b0dec2-af4e-4126-9d72-c793f70107ae": { 2024-09-16T02:05:44.057 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-16T02:05:44.057 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "00b0dec2-af4e-4126-9d72-c793f70107ae", 2024-09-16T02:05:44.057 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-16T02:05:44.057 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:05:44.057 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:05:44.058 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:05:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 -- 172.21.15.103:0/150129060 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9db00a3350 msgr2=0x7f9db00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9db00a3350 0x7f9db00a5810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0005fd0 tx=0x7f9dc0009600 comp rx=0 tx=0).stop 2024-09-16T02:05:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 -- 172.21.15.103:0/150129060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc8101de0 msgr2=0x7f9dc819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc8101de0 0x7f9dc819a6e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc004770 tx=0x7f9dbc003c10 comp rx=0 tx=0).stop 2024-09-16T02:05:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 -- 172.21.15.103:0/150129060 shutdown_connections 2024-09-16T02:05:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9db00a3350 0x7f9db00a5810 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc8106f10 0x7f9dc819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc8102790 0x7f9dc819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 --2- 172.21.15.103:0/150129060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc8101de0 0x7f9dc819a6e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 -- 172.21.15.103:0/150129060 >> 172.21.15.103:0/150129060 conn(0x7f9dc80fd8c0 msgr2=0x7f9dc80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 -- 172.21.15.103:0/150129060 shutdown_connections 2024-09-16T02:05:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:05:44.052+0000 7f9dd0400640 1 -- 172.21.15.103:0/150129060 wait complete. 2024-09-16T02:05:44.790 INFO:tasks.cephfs.filesystem:scrub status for tag:00b0dec2-af4e-4126-9d72-c793f70107ae - {'path': '/', 'tag': '00b0dec2-af4e-4126-9d72-c793f70107ae', 'options': 'recursive,force'} 2024-09-16T02:05:46.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:45 smithi103 ceph-mon[30398]: pgmap v349: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-16T02:05:46.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:45 smithi136 ceph-mon[35323]: pgmap v349: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-16T02:05:46.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:45 smithi172 ceph-mon[35340]: pgmap v349: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-16T02:05:46.995 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:46 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:05:47.001 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:46 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:05:47.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:46 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:05:48.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:47 smithi103 ceph-mon[30398]: pgmap v350: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T02:05:48.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:47 smithi136 ceph-mon[35323]: pgmap v350: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T02:05:48.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:47 smithi172 ceph-mon[35340]: pgmap v350: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T02:05:49.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:48 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:48 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:49.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:48 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:50.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:49 smithi103 ceph-mon[30398]: pgmap v351: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-16T02:05:50.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:05:50.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:05:50.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:49 smithi136 ceph-mon[35323]: pgmap v351: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-16T02:05:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:05:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:05:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:49 smithi172 ceph-mon[35340]: pgmap v351: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-16T02:05:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:05:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:05:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:05:51.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:50 smithi103 ceph-mon[30398]: pgmap v352: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T02:05:51.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:50 smithi136 ceph-mon[35323]: pgmap v352: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T02:05:51.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:50 smithi172 ceph-mon[35340]: pgmap v352: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T02:05:53.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:53 smithi172 ceph-mon[35340]: pgmap v353: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T02:05:54.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:53 smithi103 ceph-mon[30398]: pgmap v353: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T02:05:54.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:53 smithi136 ceph-mon[35323]: pgmap v353: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T02:05:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.482+0000 7fc8e2400640 1 -- 172.21.15.103:0/4064463931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8dc105910 msgr2=0x7fc8dc10c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.482+0000 7fc8e2400640 1 --2- 172.21.15.103:0/4064463931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8dc105910 0x7fc8dc10c1a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009ca0 tx=0x7fc8d402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.482+0000 7fc8e2400640 1 -- 172.21.15.103:0/4064463931 shutdown_connections 2024-09-16T02:05:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.482+0000 7fc8e2400640 1 --2- 172.21.15.103:0/4064463931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8dc105910 0x7fc8dc10c1a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.482+0000 7fc8e2400640 1 --2- 172.21.15.103:0/4064463931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8dc104f50 0x7fc8dc1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.482+0000 7fc8e2400640 1 --2- 172.21.15.103:0/4064463931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8dc103d60 0x7fc8dc104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.482+0000 7fc8e2400640 1 -- 172.21.15.103:0/4064463931 >> 172.21.15.103:0/4064463931 conn(0x7fc8dc0ff4e0 msgr2=0x7fc8dc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.482+0000 7fc8e2400640 1 -- 172.21.15.103:0/4064463931 shutdown_connections 2024-09-16T02:05:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.482+0000 7fc8e2400640 1 -- 172.21.15.103:0/4064463931 wait complete. 2024-09-16T02:05:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.483+0000 7fc8e2400640 1 Processor -- start 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.483+0000 7fc8e2400640 1 -- start start 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.483+0000 7fc8e2400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8dc103d60 0x7fc8dc19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8e2400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8dc104f50 0x7fc8dc19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8e2400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8dc105910 0x7fc8dc1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8e2400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc10fab0 con 0x7fc8dc103d60 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8e2400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc10f930 con 0x7fc8dc104f50 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8e2400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc10fc30 con 0x7fc8dc105910 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8db400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8dc103d60 0x7fc8dc19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8db400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8dc103d60 0x7fc8dc19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33630/0 (socket says 172.21.15.103:33630) 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8db400640 1 -- 172.21.15.103:0/1716411669 learned_addr learned my addr 172.21.15.103:0/1716411669 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8daa00640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8dc104f50 0x7fc8dc19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8dc105910 0x7fc8dc1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8db400640 1 -- 172.21.15.103:0/1716411669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8dc105910 msgr2=0x7fc8dc1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8db400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8dc105910 0x7fc8dc1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8db400640 1 -- 172.21.15.103:0/1716411669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8dc104f50 msgr2=0x7fc8dc19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8db400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8dc104f50 0x7fc8dc19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8db400640 1 -- 172.21.15.103:0/1716411669 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc1a1c30 con 0x7fc8dc103d60 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8dc105910 0x7fc8dc1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.484+0000 7fc8daa00640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8dc104f50 0x7fc8dc19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.485+0000 7fc8db400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8dc103d60 0x7fc8dc19a010 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00b450 tx=0x7fc8cc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.485+0000 7fc8d3e00640 1 -- 172.21.15.103:0/1716411669 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc015020 con 0x7fc8dc103d60 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.485+0000 7fc8d3e00640 1 -- 172.21.15.103:0/1716411669 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8cc011d70 con 0x7fc8dc103d60 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.485+0000 7fc8e2400640 1 -- 172.21.15.103:0/1716411669 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8dc1a1f20 con 0x7fc8dc103d60 2024-09-16T02:05:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.485+0000 7fc8d3e00640 1 -- 172.21.15.103:0/1716411669 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc00f340 con 0x7fc8dc103d60 2024-09-16T02:05:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.485+0000 7fc8e2400640 1 -- 172.21.15.103:0/1716411669 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8dc1a23b0 con 0x7fc8dc103d60 2024-09-16T02:05:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.488+0000 7fc8d3e00640 1 -- 172.21.15.103:0/1716411669 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fc8cc018020 con 0x7fc8dc103d60 2024-09-16T02:05:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.488+0000 7fc8e53d2740 1 Processor -- start 2024-09-16T02:05:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.488+0000 7fc8e53d2740 1 -- start start 2024-09-16T02:05:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9f0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 0x564f5e9f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f5a60 0x564f5e9f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f5e955100 con 0x564f5e9f0090 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564f5e8fd1b0 con 0x564f5e9f09d0 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564f5e9510c0 con 0x564f5e9f5a60 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8dbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8dbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9f0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33632/0 (socket says 172.21.15.103:33632) 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8dbe00640 1 -- 172.21.15.103:0/2881863383 learned_addr learned my addr 172.21.15.103:0/2881863383 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8db400640 1 --2- 172.21.15.103:0/2881863383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 0x564f5e9f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8daa00640 1 --2- 172.21.15.103:0/2881863383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f5a60 0x564f5e9f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8dbe00640 1 -- 172.21.15.103:0/2881863383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f5a60 msgr2=0x564f5e9f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/2881863383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f5a60 0x564f5e9f7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8dbe00640 1 -- 172.21.15.103:0/2881863383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 msgr2=0x564f5e9f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/2881863383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 0x564f5e9f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8dbe00640 1 -- 172.21.15.103:0/2881863383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f5e668210 con 0x564f5e9f0090 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.489+0000 7fc8db400640 1 --2- 172.21.15.103:0/2881863383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 0x564f5e9f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.490+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/2881863383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9f0490 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009ca0 tx=0x7fc8d4009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ee5860841f4484 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.490+0000 7fc8d3e00640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc8c00a3390 0x7fc8c00a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.490+0000 7fc8d3e00640 1 -- 172.21.15.103:0/1716411669 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fc8cc0cf580 con 0x7fc8dc103d60 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.490+0000 7fc8d0c00640 1 -- 172.21.15.103:0/2881863383 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4031790 con 0x564f5e9f0090 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.490+0000 7fc8d0c00640 1 -- 172.21.15.103:0/2881863383 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d4036040 con 0x564f5e9f0090 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.490+0000 7fc8d0c00640 1 -- 172.21.15.103:0/2881863383 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d402f610 con 0x564f5e9f0090 2024-09-16T02:05:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.490+0000 7fc8db400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc8c00a3390 0x7fc8c00a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2881863383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 msgr2=0x564f5e9f0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8db400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc8c00a3390 0x7fc8c00a5850 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00be10 tx=0x7fc8cc0047d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/2881863383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9f0490 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009ca0 tx=0x7fc8d4009910 comp rx=0 tx=0).stop 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2881863383 shutdown_connections 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/2881863383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f5a60 0x564f5e9f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/2881863383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 0x564f5e9f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/2881863383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9f0490 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2881863383 >> 172.21.15.103:0/2881863383 conn(0x564f5e9eb700 msgr2=0x564f5e9ebb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2881863383 shutdown_connections 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2881863383 wait complete. 2024-09-16T02:05:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.491+0000 7fc8e53d2740 1 -- start start 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8e53d2740 1 Processor -- start 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9fa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f09d0 0x564f5e9fa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f5a60 0x564f5ea01790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f5e9f89d0 con 0x564f5e9f0090 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564f5e8de090 con 0x564f5e9f5a60 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564f5e9f8b50 con 0x564f5e9f09d0 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8e53d2740 10 client.0 _ll_register_callbacks cb 0x564f5e932f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8dbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8dbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9fa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33636/0 (socket says 172.21.15.103:33636) 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.492+0000 7fc8dbe00640 1 -- 172.21.15.103:0/3354147086 learned_addr learned my addr 172.21.15.103:0/3354147086 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.493+0000 7fc8dbe00640 1 -- 172.21.15.103:0/3354147086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f09d0 msgr2=0x564f5e9fa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.493+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f09d0 0x564f5e9fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.493+0000 7fc8dbe00640 1 -- 172.21.15.103:0/3354147086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f5a60 msgr2=0x564f5ea01790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.493+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f5a60 0x564f5ea01790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.493+0000 7fc8dbe00640 1 -- 172.21.15.103:0/3354147086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f5e734620 con 0x564f5e9f0090 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.493+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9fa210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009430 tx=0x7fc8d40048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9891b0a6a784ed51 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.493+0000 7fc8c5600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.493+0000 7fc8c5600640 1 -- 172.21.15.103:0/3354147086 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4053020 con 0x564f5e9f0090 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.493+0000 7fc8c5600640 1 -- 172.21.15.103:0/3354147086 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fc8d4031690 con 0x564f5e9f0090 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.493+0000 7fc8e53d2740 1 -- 172.21.15.103:0/3354147086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d4036040 con 0x564f5e9f0090 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8c5600640 1 -- 172.21.15.103:0/3354147086 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc8d405e050 con 0x564f5e9f0090 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8e53d2740 10 client.15168 fetch_fsmap learned FSMap version 24 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8e53d2740 1 -- 172.21.15.103:0/3354147086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564f5e9f89d0 con 0x564f5e9f0090 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8c5600640 1 -- 172.21.15.103:0/3354147086 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d4036040 con 0x564f5e9f0090 2024-09-16T02:05:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8c5600640 1 -- 172.21.15.103:0/3354147086 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc8d4030750 con 0x564f5e9f0090 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8e53d2740 10 client.15168 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8e53d2740 10 client.15168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8e53d2740 20 client.15168 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x564f5ea05910 0x564f5ea07d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8e53d2740 4 client.15168 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.494+0000 7fc8e53d2740 1 -- 172.21.15.103:0/3354147086 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f5e962830 con 0x564f5ea05910 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.495+0000 7fc8daa00640 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x564f5ea05910 0x564f5ea07d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.495+0000 7fc8daa00640 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x564f5ea05910 0x564f5ea07d00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58cf2cc0afcb147c server_cookie=d4e2338ec9c87f31 in_seq=0 out_seq=0 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.495+0000 7fc8c5600640 10 client.15168 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8c5600640 1 -- 172.21.15.103:0/3354147086 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f5e962830 con 0x564f5ea05910 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8c5600640 10 client.15168 handle_command_reply: tid=0 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8e53d2740 1 client.15168 shutdown 2024-09-16T02:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8e53d2740 1 -- 172.21.15.103:0/3354147086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 msgr2=0x564f5e9fa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9fa210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009430 tx=0x7fc8d40048c0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8e53d2740 1 -- 172.21.15.103:0/3354147086 shutdown_connections 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x564f5ea05910 0x564f5ea07d00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f5a60 0x564f5ea01790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f09d0 0x564f5e9fa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/3354147086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f0090 0x564f5e9fa210 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8c5600640 0 client.15168 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.496+0000 7fc8e53d2740 1 -- 172.21.15.103:0/3354147086 >> 172.21.15.103:0/3354147086 conn(0x564f5e9eb700 msgr2=0x564f5e9ecf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.497+0000 7fc8e53d2740 1 -- 172.21.15.103:0/3354147086 shutdown_connections 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.497+0000 7fc8e53d2740 1 -- 172.21.15.103:0/3354147086 wait complete. 2024-09-16T02:05:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.497+0000 7fc8e53d2740 20 client.15168 trim_cache size 0 max 16384 2024-09-16T02:05:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8e53d2740 1 Processor -- start 2024-09-16T02:05:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8e53d2740 1 -- start start 2024-09-16T02:05:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f0090 0x564f5e9fa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 0x564f5ea01750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f5a60 0x564f5e9ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1add40 con 0x564f5e9f5a60 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ab610 con 0x564f5e9f09d0 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1a9460 con 0x564f5e9f0090 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8dbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f5a60 0x564f5e9ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8dbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f5a60 0x564f5e9ec5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33650/0 (socket says 172.21.15.103:33650) 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8dbe00640 1 -- 172.21.15.103:0/768845910 learned_addr learned my addr 172.21.15.103:0/768845910 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.512+0000 7fc8daa00640 1 --2- 172.21.15.103:0/768845910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 0x564f5ea01750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.513+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/768845910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f0090 0x564f5e9fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.513+0000 7fc8dbe00640 1 -- 172.21.15.103:0/768845910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f0090 msgr2=0x564f5e9fa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.513+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/768845910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f0090 0x564f5e9fa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.513+0000 7fc8dbe00640 1 -- 172.21.15.103:0/768845910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 msgr2=0x564f5ea01750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.513+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/768845910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 0x564f5ea01750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.513+0000 7fc8dbe00640 1 -- 172.21.15.103:0/768845910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f5ea08380 con 0x564f5e9f5a60 2024-09-16T02:05:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.513+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/768845910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f5a60 0x564f5e9ec5b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40093c0 tx=0x7fc8d4009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30fd5c33cfa4e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.513+0000 7fc8c7e00640 1 -- 172.21.15.103:0/768845910 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4053040 con 0x564f5e9f5a60 2024-09-16T02:05:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.513+0000 7fc8c7e00640 1 -- 172.21.15.103:0/768845910 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d4036070 con 0x564f5e9f5a60 2024-09-16T02:05:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.513+0000 7fc8c7e00640 1 -- 172.21.15.103:0/768845910 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d402f990 con 0x564f5e9f5a60 2024-09-16T02:05:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.514+0000 7fc8e53d2740 1 -- 172.21.15.103:0/768845910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f5a60 msgr2=0x564f5e9ec5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.514+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/768845910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f5a60 0x564f5e9ec5b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40093c0 tx=0x7fc8d4009910 comp rx=0 tx=0).stop 2024-09-16T02:05:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.514+0000 7fc8e53d2740 1 -- 172.21.15.103:0/768845910 shutdown_connections 2024-09-16T02:05:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.514+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/768845910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f5a60 0x564f5e9ec5b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.514+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/768845910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f09d0 0x564f5ea01750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.514+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/768845910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f0090 0x564f5e9fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.514+0000 7fc8e53d2740 1 -- 172.21.15.103:0/768845910 >> 172.21.15.103:0/768845910 conn(0x564f5e9eb700 msgr2=0x564f5e9f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.514+0000 7fc8e53d2740 1 -- 172.21.15.103:0/768845910 shutdown_connections 2024-09-16T02:05:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.514+0000 7fc8e53d2740 1 -- 172.21.15.103:0/768845910 wait complete. 2024-09-16T02:05:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.514+0000 7fc8e53d2740 1 -- start start 2024-09-16T02:05:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8e53d2740 1 Processor -- start 2024-09-16T02:05:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f0090 0x564f5e9fe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f09d0 0x564f5e9feea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8e53d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f5a60 0x564f5e9ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1a9ad0 con 0x564f5e9f09d0 2024-09-16T02:05:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1aaf40 con 0x564f5e9f5a60 2024-09-16T02:05:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8e53d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1aa1a0 con 0x564f5e9f0090 2024-09-16T02:05:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8e53d2740 10 client.0 _ll_register_callbacks cb 0x7fc8d4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:05:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8daa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f09d0 0x564f5e9feea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8daa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f09d0 0x564f5e9feea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33660/0 (socket says 172.21.15.103:33660) 2024-09-16T02:05:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.515+0000 7fc8daa00640 1 -- 172.21.15.103:0/2169498763 learned_addr learned my addr 172.21.15.103:0/2169498763 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:05:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8daa00640 1 -- 172.21.15.103:0/2169498763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f0090 msgr2=0x564f5e9fe960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8daa00640 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f0090 0x564f5e9fe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8daa00640 1 -- 172.21.15.103:0/2169498763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f5a60 msgr2=0x564f5e9ffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8daa00640 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f5a60 0x564f5e9ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8daa00640 1 -- 172.21.15.103:0/2169498763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f5ea004c0 con 0x564f5e9f09d0 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8daa00640 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f09d0 0x564f5e9feea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8000c00 tx=0x7fc8c8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b675863ae1bcb34 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8c5600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8c5600640 1 -- 172.21.15.103:0/2169498763 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8003d20 con 0x564f5e9f09d0 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2169498763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d4036070 con 0x564f5e9f09d0 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8c5600640 1 -- 172.21.15.103:0/2169498763 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fc8c801f390 con 0x564f5e9f09d0 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.516+0000 7fc8c5600640 1 -- 172.21.15.103:0/2169498763 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc8c802c020 con 0x564f5e9f09d0 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.517+0000 7fc8e53d2740 10 client.15174 fetch_fsmap learned FSMap version 24 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.517+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2169498763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564f5ea01920 con 0x564f5e9f09d0 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.517+0000 7fc8c5600640 1 -- 172.21.15.103:0/2169498763 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d4036070 con 0x564f5e9f09d0 2024-09-16T02:05:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.517+0000 7fc8c5600640 1 -- 172.21.15.103:0/2169498763 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc8c8012870 con 0x564f5e9f09d0 2024-09-16T02:05:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.517+0000 7fc8e53d2740 10 client.15174 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:05:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.517+0000 7fc8e53d2740 10 client.15174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:05:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.517+0000 7fc8e53d2740 20 client.15174 populate_metadata read hostname 'smithi103' 2024-09-16T02:05:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.517+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x564f5ea70b90 0x564f5ea72fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:05:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.517+0000 7fc8e53d2740 4 client.15174 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T02:05:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.517+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2169498763 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f5e9f57e0 con 0x564f5ea70b90 2024-09-16T02:05:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.518+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x564f5ea70b90 0x564f5ea72fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:05:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.519+0000 7fc8dbe00640 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x564f5ea70b90 0x564f5ea72fb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ab233f1486972 server_cookie=ae0c49fd996cdb7f in_seq=0 out_seq=0 2024-09-16T02:05:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.519+0000 7fc8c5600640 10 client.15174 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.531+0000 7fc8c5600640 1 -- 172.21.15.103:0/2169498763 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f5e9f57e0 con 0x564f5ea70b90 2024-09-16T02:05:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.531+0000 7fc8c5600640 10 client.15174 handle_command_reply: tid=0 2024-09-16T02:05:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.531+0000 7fc8e53d2740 1 client.15174 shutdown 2024-09-16T02:05:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2169498763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f09d0 msgr2=0x564f5e9feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f09d0 0x564f5e9feea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8000c00 tx=0x7fc8c8004410 comp rx=0 tx=0).stop 2024-09-16T02:05:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2169498763 shutdown_connections 2024-09-16T02:05:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x564f5ea70b90 0x564f5ea72fb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f5e9f5a60 0x564f5e9ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f5e9f09d0 0x564f5e9feea0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8c5600640 0 client.15174 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:05:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 1 --2- 172.21.15.103:0/2169498763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f5e9f0090 0x564f5e9fe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2169498763 >> 172.21.15.103:0/2169498763 conn(0x564f5e9eb700 msgr2=0x564f5e9fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2169498763 shutdown_connections 2024-09-16T02:05:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 1 -- 172.21.15.103:0/2169498763 wait complete. 2024-09-16T02:05:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.532+0000 7fc8e53d2740 20 client.15174 trim_cache size 0 max 16384 2024-09-16T02:05:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-16T02:05:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-16T02:05:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-16T02:05:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-16T02:05:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 -- 172.21.15.103:0/1716411669 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc8c00a3390 msgr2=0x7fc8c00a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc8c00a3390 0x7fc8c00a5850 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00be10 tx=0x7fc8cc0047d0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 -- 172.21.15.103:0/1716411669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8dc103d60 msgr2=0x7fc8dc19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:05:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8dc103d60 0x7fc8dc19a010 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00b450 tx=0x7fc8cc00b910 comp rx=0 tx=0).stop 2024-09-16T02:05:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 -- 172.21.15.103:0/1716411669 shutdown_connections 2024-09-16T02:05:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc8c00a3390 0x7fc8c00a5850 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8dc105910 0x7fc8dc1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8dc104f50 0x7fc8dc19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 --2- 172.21.15.103:0/1716411669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8dc103d60 0x7fc8dc19a010 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:05:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 -- 172.21.15.103:0/1716411669 >> 172.21.15.103:0/1716411669 conn(0x7fc8dc0ff4e0 msgr2=0x7fc8dc1081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:05:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 -- 172.21.15.103:0/1716411669 shutdown_connections 2024-09-16T02:05:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:05:54.533+0000 7fc8e2400640 1 -- 172.21.15.103:0/1716411669 wait complete. 2024-09-16T02:05:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:55 smithi172 ceph-mon[35340]: pgmap v354: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T02:05:55.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:55 smithi172 ceph-mon[35340]: scrub summary: idle 2024-09-16T02:05:56.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:55 smithi103 ceph-mon[30398]: pgmap v354: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T02:05:56.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:55 smithi103 ceph-mon[30398]: scrub summary: idle 2024-09-16T02:05:56.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:55 smithi136 ceph-mon[35323]: pgmap v354: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T02:05:56.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:55 smithi136 ceph-mon[35323]: scrub summary: idle 2024-09-16T02:05:58.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:57 smithi103 ceph-mon[30398]: pgmap v355: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T02:05:58.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:57 smithi136 ceph-mon[35323]: pgmap v355: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T02:05:58.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:57 smithi172 ceph-mon[35340]: pgmap v355: 97 pgs: 97 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T02:06:00.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:05:59 smithi103 ceph-mon[30398]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-16T02:06:00.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:05:59 smithi136 ceph-mon[35323]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-16T02:06:00.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:05:59 smithi172 ceph-mon[35340]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-16T02:06:02.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:01 smithi103 ceph-mon[30398]: pgmap v357: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-16T02:06:02.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:01 smithi136 ceph-mon[35323]: pgmap v357: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-16T02:06:02.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:01 smithi172 ceph-mon[35340]: pgmap v357: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-16T02:06:04.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:03 smithi103 ceph-mon[30398]: pgmap v358: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-16T02:06:04.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:03 smithi136 ceph-mon[35323]: pgmap v358: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-16T02:06:04.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:03 smithi172 ceph-mon[35340]: pgmap v358: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-16T02:06:06.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:05 smithi103 ceph-mon[30398]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-16T02:06:06.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:05 smithi136 ceph-mon[35323]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-16T02:06:06.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:05 smithi172 ceph-mon[35340]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-16T02:06:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.650+0000 7fab86c00640 1 -- 172.21.15.103:0/4262421750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab80104f50 msgr2=0x7fab801053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.650+0000 7fab86c00640 1 --2- 172.21.15.103:0/4262421750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab80104f50 0x7fab801053d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009900 tx=0x7fab6c02a410 comp rx=0 tx=0).stop 2024-09-16T02:06:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.650+0000 7fab86c00640 1 -- 172.21.15.103:0/4262421750 shutdown_connections 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.650+0000 7fab86c00640 1 --2- 172.21.15.103:0/4262421750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab80105910 0x7fab8010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.650+0000 7fab86c00640 1 --2- 172.21.15.103:0/4262421750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab80104f50 0x7fab801053d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.650+0000 7fab86c00640 1 --2- 172.21.15.103:0/4262421750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab80103d60 0x7fab80104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.650+0000 7fab86c00640 1 -- 172.21.15.103:0/4262421750 >> 172.21.15.103:0/4262421750 conn(0x7fab800ff4e0 msgr2=0x7fab80101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.651+0000 7fab86c00640 1 -- 172.21.15.103:0/4262421750 shutdown_connections 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.651+0000 7fab86c00640 1 -- 172.21.15.103:0/4262421750 wait complete. 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.651+0000 7fab86c00640 1 Processor -- start 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab86c00640 1 -- start start 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab86c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab80103d60 0x7fab80199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab86c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab80104f50 0x7fab8019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab86c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab80105910 0x7fab801a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab86c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab8010fa30 con 0x7fab80105910 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab86c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab8010f8b0 con 0x7fab80104f50 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab86c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fab8010fbb0 con 0x7fab80103d60 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab7fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab80103d60 0x7fab80199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab7fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab80103d60 0x7fab80199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:36148/0 (socket says 172.21.15.103:36148) 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab7fe00640 1 -- 172.21.15.103:0/1396255680 learned_addr learned my addr 172.21.15.103:0/1396255680 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab84c00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab80105910 0x7fab801a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.652+0000 7fab7f400640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab80104f50 0x7fab8019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.653+0000 7fab7fe00640 1 -- 172.21.15.103:0/1396255680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab80104f50 msgr2=0x7fab8019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.653+0000 7fab7fe00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab80104f50 0x7fab8019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.653+0000 7fab7fe00640 1 -- 172.21.15.103:0/1396255680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab80105910 msgr2=0x7fab801a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.653+0000 7fab7fe00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab80105910 0x7fab801a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.653+0000 7fab7fe00640 1 -- 172.21.15.103:0/1396255680 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab801a1bc0 con 0x7fab80103d60 2024-09-16T02:06:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.653+0000 7fab7fe00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab80103d60 0x7fab80199fa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab7000e9c0 tx=0x7fab7000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.653+0000 7fab7cc00640 1 -- 172.21.15.103:0/1396255680 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7000cd80 con 0x7fab80103d60 2024-09-16T02:06:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.653+0000 7fab7cc00640 1 -- 172.21.15.103:0/1396255680 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab700035f0 con 0x7fab80103d60 2024-09-16T02:06:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.653+0000 7fab86c00640 1 -- 172.21.15.103:0/1396255680 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab801a1eb0 con 0x7fab80103d60 2024-09-16T02:06:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.653+0000 7fab7cc00640 1 -- 172.21.15.103:0/1396255680 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70010690 con 0x7fab80103d60 2024-09-16T02:06:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.654+0000 7fab86c00640 1 -- 172.21.15.103:0/1396255680 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab801a2320 con 0x7fab80103d60 2024-09-16T02:06:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.657+0000 7fab89a7c740 1 Processor -- start 2024-09-16T02:06:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab89a7c740 1 -- start start 2024-09-16T02:06:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c151a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 0x5623c151f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c1521f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab89a7c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5623c147f100 con 0x5623c151fa60 2024-09-16T02:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab89a7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5623c14271b0 con 0x5623c151a090 2024-09-16T02:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab89a7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5623c147b0c0 con 0x5623c151a9d0 2024-09-16T02:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab7cc00640 1 -- 172.21.15.103:0/1396255680 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fab700037a0 con 0x7fab80103d60 2024-09-16T02:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab7f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c1521f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab7f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c1521f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59862/0 (socket says 172.21.15.103:59862) 2024-09-16T02:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab7f400640 1 -- 172.21.15.103:0/356401388 learned_addr learned my addr 172.21.15.103:0/356401388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.658+0000 7fab84c00640 1 --2- 172.21.15.103:0/356401388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c151a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.659+0000 7fab7fe00640 1 --2- 172.21.15.103:0/356401388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 0x5623c151f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.659+0000 7fab84c00640 1 -- 172.21.15.103:0/356401388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 msgr2=0x5623c151f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.659+0000 7fab84c00640 1 --2- 172.21.15.103:0/356401388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 0x5623c151f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.659+0000 7fab84c00640 1 -- 172.21.15.103:0/356401388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 msgr2=0x5623c1521f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.659+0000 7fab84c00640 1 --2- 172.21.15.103:0/356401388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c1521f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.659+0000 7fab84c00640 1 -- 172.21.15.103:0/356401388 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623c1192210 con 0x5623c151a090 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.659+0000 7fab84c00640 1 --2- 172.21.15.103:0/356401388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c151a490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab78009f00 tx=0x7fab78011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57aa1b2a6313fa4e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.659+0000 7fab75200640 1 -- 172.21.15.103:0/356401388 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78004d20 con 0x5623c151a090 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.659+0000 7fab75200640 1 -- 172.21.15.103:0/356401388 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab78004030 con 0x5623c151a090 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.659+0000 7fab75200640 1 -- 172.21.15.103:0/356401388 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78005260 con 0x5623c151a090 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab7cc00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fab640a3410 0x7fab640a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab7cc00640 1 -- 172.21.15.103:0/1396255680 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fab700cfef0 con 0x7fab80103d60 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab89a7c740 1 -- 172.21.15.103:0/356401388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 msgr2=0x5623c151a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab89a7c740 1 --2- 172.21.15.103:0/356401388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c151a490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab78009f00 tx=0x7fab78011810 comp rx=0 tx=0).stop 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab7fe00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fab640a3410 0x7fab640a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab89a7c740 1 -- 172.21.15.103:0/356401388 shutdown_connections 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab89a7c740 1 --2- 172.21.15.103:0/356401388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c1521f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab89a7c740 1 --2- 172.21.15.103:0/356401388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 0x5623c151f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab89a7c740 1 --2- 172.21.15.103:0/356401388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c151a490 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab89a7c740 1 -- 172.21.15.103:0/356401388 >> 172.21.15.103:0/356401388 conn(0x5623c1515700 msgr2=0x5623c1515b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab89a7c740 1 -- 172.21.15.103:0/356401388 shutdown_connections 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.660+0000 7fab89a7c740 1 -- 172.21.15.103:0/356401388 wait complete. 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.661+0000 7fab7fe00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fab640a3410 0x7fab640a58d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab7009c260 tx=0x7fab70000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.661+0000 7fab89a7c740 1 -- start start 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.661+0000 7fab89a7c740 1 Processor -- start 2024-09-16T02:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.661+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c1524220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.661+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 0x5623c1524760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.661+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c152b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab89a7c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5623c1522a30 con 0x5623c151fa60 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab89a7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5623c1408090 con 0x5623c151a090 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab89a7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5623c1522bb0 con 0x5623c151a9d0 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab89a7c740 10 client.0 _ll_register_callbacks cb 0x5623c145cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab84c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c152b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab84c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c152b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59878/0 (socket says 172.21.15.103:59878) 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab84c00640 1 -- 172.21.15.103:0/1835866013 learned_addr learned my addr 172.21.15.103:0/1835866013 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab7f400640 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 0x5623c1524760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab84c00640 1 -- 172.21.15.103:0/1835866013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 msgr2=0x5623c1524760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab84c00640 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 0x5623c1524760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab84c00640 1 -- 172.21.15.103:0/1835866013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 msgr2=0x5623c1524220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab84c00640 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c1524220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab84c00640 1 -- 172.21.15.103:0/1835866013 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623c125e620 con 0x5623c151fa60 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.662+0000 7fab7f400640 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 0x5623c1524760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.663+0000 7fab84c00640 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c152b7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fab78009d40 tx=0x7fab78004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c460782a217aa87 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.663+0000 7fab69600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.663+0000 7fab69600640 1 -- 172.21.15.103:0/1835866013 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78005890 con 0x5623c151fa60 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.663+0000 7fab69600640 1 -- 172.21.15.103:0/1835866013 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fab78004a40 con 0x5623c151fa60 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.663+0000 7fab89a7c740 1 -- 172.21.15.103:0/1835866013 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab78004030 con 0x5623c151fa60 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.663+0000 7fab69600640 1 -- 172.21.15.103:0/1835866013 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fab78004c40 con 0x5623c151fa60 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.663+0000 7fab89a7c740 10 client.15186 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.663+0000 7fab89a7c740 1 -- 172.21.15.103:0/1835866013 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5623c152e590 con 0x5623c151fa60 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.663+0000 7fab69600640 1 -- 172.21.15.103:0/1835866013 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab78004030 con 0x5623c151fa60 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.664+0000 7fab69600640 1 -- 172.21.15.103:0/1835866013 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fab78018070 con 0x5623c151fa60 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.664+0000 7fab89a7c740 10 client.15186 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.664+0000 7fab89a7c740 10 client.15186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.664+0000 7fab89a7c740 20 client.15186 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.664+0000 7fab89a7c740 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5623c152fab0 0x5623c1531f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.664+0000 7fab89a7c740 4 client.15186 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.664+0000 7fab89a7c740 1 -- 172.21.15.103:0/1835866013 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623c148c830 con 0x5623c152fab0 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.664+0000 7fab7f400640 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5623c152fab0 0x5623c1531f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.665+0000 7fab7f400640 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5623c152fab0 0x5623c1531f10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44c3e7ffa05a1b7d server_cookie=c4b9355aa3da15f0 in_seq=0 out_seq=0 2024-09-16T02:06:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.665+0000 7fab69600640 10 client.15186 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab69600640 1 -- 172.21.15.103:0/1835866013 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623c148c830 con 0x5623c152fab0 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab69600640 10 client.15186 handle_command_reply: tid=0 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 client.15186 shutdown 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 -- 172.21.15.103:0/1835866013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 msgr2=0x5623c152b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c152b7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fab78009d40 tx=0x7fab78004410 comp rx=0 tx=0).stop 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 -- 172.21.15.103:0/1835866013 shutdown_connections 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5623c152fab0 0x5623c1531f10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151fa60 0x5623c152b7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151a9d0 0x5623c1524760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 --2- 172.21.15.103:0/1835866013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c1524220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 -- 172.21.15.103:0/1835866013 >> 172.21.15.103:0/1835866013 conn(0x5623c1515700 msgr2=0x5623c1516f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab69600640 0 client.15186 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 -- 172.21.15.103:0/1835866013 shutdown_connections 2024-09-16T02:06:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 1 -- 172.21.15.103:0/1835866013 wait complete. 2024-09-16T02:06:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.666+0000 7fab89a7c740 20 client.15186 trim_cache size 0 max 16384 2024-09-16T02:06:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab89a7c740 1 Processor -- start 2024-09-16T02:06:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab89a7c740 1 -- start start 2024-09-16T02:06:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a090 0x5623c1523690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a9d0 0x5623c152b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151fa60 0x5623c15165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab89a7c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab801adc50 con 0x5623c151a090 2024-09-16T02:06:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab89a7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab801ab520 con 0x5623c151a9d0 2024-09-16T02:06:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab89a7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fab801a9370 con 0x5623c151fa60 2024-09-16T02:06:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab84c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a090 0x5623c1523690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab84c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a090 0x5623c1523690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59882/0 (socket says 172.21.15.103:59882) 2024-09-16T02:06:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab84c00640 1 -- 172.21.15.103:0/3658716936 learned_addr learned my addr 172.21.15.103:0/3658716936 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.683+0000 7fab7f400640 1 --2- 172.21.15.103:0/3658716936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a9d0 0x5623c152b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.684+0000 7fab84c00640 1 --2- 172.21.15.103:0/3658716936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151fa60 0x5623c15165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.684+0000 7fab7f400640 1 -- 172.21.15.103:0/3658716936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151fa60 msgr2=0x5623c15165b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.684+0000 7fab7f400640 1 --2- 172.21.15.103:0/3658716936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151fa60 0x5623c15165b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.684+0000 7fab7f400640 1 -- 172.21.15.103:0/3658716936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a090 msgr2=0x5623c1523690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.684+0000 7fab7f400640 1 --2- 172.21.15.103:0/3658716936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a090 0x5623c1523690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.684+0000 7fab7f400640 1 -- 172.21.15.103:0/3658716936 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623c151b660 con 0x5623c151a9d0 2024-09-16T02:06:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.684+0000 7fab84c00640 1 --2- 172.21.15.103:0/3658716936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a090 0x5623c1523690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.684+0000 7fab7f400640 1 --2- 172.21.15.103:0/3658716936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a9d0 0x5623c152b710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab6c006010 tx=0x7fab6c0315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5b6119cb1a83c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab6be00640 1 -- 172.21.15.103:0/3658716936 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c04a070 con 0x5623c151a9d0 2024-09-16T02:06:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab6be00640 1 -- 172.21.15.103:0/3658716936 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6c005530 con 0x5623c151a9d0 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab6be00640 1 -- 172.21.15.103:0/3658716936 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c035040 con 0x5623c151a9d0 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab89a7c740 1 -- 172.21.15.103:0/3658716936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a9d0 msgr2=0x5623c152b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab89a7c740 1 --2- 172.21.15.103:0/3658716936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a9d0 0x5623c152b710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab6c006010 tx=0x7fab6c0315b0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab89a7c740 1 -- 172.21.15.103:0/3658716936 shutdown_connections 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab89a7c740 1 --2- 172.21.15.103:0/3658716936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151fa60 0x5623c15165b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab89a7c740 1 --2- 172.21.15.103:0/3658716936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a9d0 0x5623c152b710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab89a7c740 1 --2- 172.21.15.103:0/3658716936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a090 0x5623c1523690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab89a7c740 1 -- 172.21.15.103:0/3658716936 >> 172.21.15.103:0/3658716936 conn(0x5623c1515700 msgr2=0x5623c1522f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab89a7c740 1 -- 172.21.15.103:0/3658716936 shutdown_connections 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.685+0000 7fab89a7c740 1 -- 172.21.15.103:0/3658716936 wait complete. 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.686+0000 7fab89a7c740 1 -- start start 2024-09-16T02:06:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.686+0000 7fab89a7c740 1 Processor -- start 2024-09-16T02:06:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.686+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c1528250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.686+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a9d0 0x5623c1528790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.686+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151fa60 0x5623c15987a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.686+0000 7fab89a7c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab801a99e0 con 0x5623c151a9d0 2024-09-16T02:06:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.686+0000 7fab89a7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab801aae50 con 0x5623c151a090 2024-09-16T02:06:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.686+0000 7fab89a7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fab801aa0b0 con 0x5623c151fa60 2024-09-16T02:06:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.686+0000 7fab89a7c740 10 client.0 _ll_register_callbacks cb 0x7fab780093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab7f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a9d0 0x5623c1528790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab7f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a9d0 0x5623c1528790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59890/0 (socket says 172.21.15.103:59890) 2024-09-16T02:06:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab7f400640 1 -- 172.21.15.103:0/481645109 learned_addr learned my addr 172.21.15.103:0/481645109 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab84c00640 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c1528250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab84c00640 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151fa60 0x5623c15987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab7f400640 1 -- 172.21.15.103:0/481645109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151fa60 msgr2=0x5623c15987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab7f400640 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151fa60 0x5623c15987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab7f400640 1 -- 172.21.15.103:0/481645109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 msgr2=0x5623c1528250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab7f400640 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c1528250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab7f400640 1 -- 172.21.15.103:0/481645109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623c152b830 con 0x5623c151a9d0 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab7f400640 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a9d0 0x5623c1528790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0314e0 tx=0x7fab6c0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356eb47e185d6087 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.687+0000 7fab69600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.688+0000 7fab69600640 1 -- 172.21.15.103:0/481645109 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c035070 con 0x5623c151a9d0 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.688+0000 7fab69600640 1 -- 172.21.15.103:0/481645109 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fab6c004ce0 con 0x5623c151a9d0 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.688+0000 7fab89a7c740 1 -- 172.21.15.103:0/481645109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab6c005530 con 0x5623c151a9d0 2024-09-16T02:06:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.688+0000 7fab69600640 1 -- 172.21.15.103:0/481645109 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fab6c054020 con 0x5623c151a9d0 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.688+0000 7fab89a7c740 10 client.15192 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.688+0000 7fab89a7c740 1 -- 172.21.15.103:0/481645109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5623c1599bd0 con 0x5623c151a9d0 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.688+0000 7fab69600640 1 -- 172.21.15.103:0/481645109 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6c005530 con 0x5623c151a9d0 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.689+0000 7fab69600640 1 -- 172.21.15.103:0/481645109 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fab6c031be0 con 0x5623c151a9d0 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.689+0000 7fab89a7c740 10 client.15192 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.689+0000 7fab89a7c740 10 client.15192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.689+0000 7fab89a7c740 20 client.15192 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.689+0000 7fab89a7c740 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5623c159a4c0 0x5623c159c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.689+0000 7fab89a7c740 4 client.15192 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.689+0000 7fab89a7c740 1 -- 172.21.15.103:0/481645109 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623c1486b80 con 0x5623c159a4c0 2024-09-16T02:06:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.689+0000 7fab84c00640 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5623c159a4c0 0x5623c159c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.691+0000 7fab69600640 10 client.15192 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.691+0000 7fab84c00640 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5623c159a4c0 0x5623c159c8e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83acdde58586e773 server_cookie=7cc41a696a356738 in_seq=0 out_seq=0 2024-09-16T02:06:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab69600640 1 -- 172.21.15.103:0/481645109 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623c1486b80 con 0x5623c159a4c0 2024-09-16T02:06:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab69600640 10 client.15192 handle_command_reply: tid=0 2024-09-16T02:06:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 client.15192 shutdown 2024-09-16T02:06:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 -- 172.21.15.103:0/481645109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a9d0 msgr2=0x5623c1528790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a9d0 0x5623c1528790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0314e0 tx=0x7fab6c0056e0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 -- 172.21.15.103:0/481645109 shutdown_connections 2024-09-16T02:06:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5623c159a4c0 0x5623c159c8e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623c151fa60 0x5623c15987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c151a9d0 0x5623c1528790 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 --2- 172.21.15.103:0/481645109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623c151a090 0x5623c1528250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab69600640 0 client.15192 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 -- 172.21.15.103:0/481645109 >> 172.21.15.103:0/481645109 conn(0x5623c1515700 msgr2=0x5623c15165b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 -- 172.21.15.103:0/481645109 shutdown_connections 2024-09-16T02:06:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 1 -- 172.21.15.103:0/481645109 wait complete. 2024-09-16T02:06:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.699+0000 7fab89a7c740 20 client.15192 trim_cache size 0 max 16384 2024-09-16T02:06:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-16T02:06:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-16T02:06:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-16T02:06:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-16T02:06:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 -- 172.21.15.103:0/1396255680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fab640a3410 msgr2=0x7fab640a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fab640a3410 0x7fab640a58d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab7009c260 tx=0x7fab70000bc0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 -- 172.21.15.103:0/1396255680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab80103d60 msgr2=0x7fab80199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab80103d60 0x7fab80199fa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab7000e9c0 tx=0x7fab7000ee80 comp rx=0 tx=0).stop 2024-09-16T02:06:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 -- 172.21.15.103:0/1396255680 shutdown_connections 2024-09-16T02:06:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fab640a3410 0x7fab640a58d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab80105910 0x7fab801a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab80104f50 0x7fab8019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 --2- 172.21.15.103:0/1396255680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab80103d60 0x7fab80199fa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 -- 172.21.15.103:0/1396255680 >> 172.21.15.103:0/1396255680 conn(0x7fab800ff4e0 msgr2=0x7fab80108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 -- 172.21.15.103:0/1396255680 shutdown_connections 2024-09-16T02:06:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:07.700+0000 7fab86c00640 1 -- 172.21.15.103:0/1396255680 wait complete. 2024-09-16T02:06:07.848 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:07 smithi172 ceph-mon[35340]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-16T02:06:08.008 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:07 smithi103 ceph-mon[30398]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-16T02:06:08.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:07 smithi136 ceph-mon[35323]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-16T02:06:09.825 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:09 smithi136 ceph-mon[35323]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-16T02:06:10.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:09 smithi103 ceph-mon[30398]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-16T02:06:10.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:09 smithi172 ceph-mon[35340]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-16T02:06:12.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:11 smithi103 ceph-mon[30398]: pgmap v362: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T02:06:12.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:11 smithi136 ceph-mon[35323]: pgmap v362: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T02:06:12.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:11 smithi172 ceph-mon[35340]: pgmap v362: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T02:06:14.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:13 smithi103 ceph-mon[30398]: pgmap v363: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T02:06:14.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:13 smithi136 ceph-mon[35323]: pgmap v363: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T02:06:14.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:13 smithi172 ceph-mon[35340]: pgmap v363: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-16T02:06:14.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:06:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:15.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:15 smithi103 ceph-mon[30398]: pgmap v364: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-16T02:06:15.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.950+0000 7fc4d7e00640 1 -- 172.21.15.103:0/4267122251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4d81034b0 msgr2=0x7fc4d810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:15.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.950+0000 7fc4d7e00640 1 --2- 172.21.15.103:0/4267122251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4d81034b0 0x7fc4d810b9a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009900 tx=0x7fc4c002a480 comp rx=0 tx=0).stop 2024-09-16T02:06:15.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.950+0000 7fc4d7e00640 1 -- 172.21.15.103:0/4267122251 shutdown_connections 2024-09-16T02:06:15.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.950+0000 7fc4d7e00640 1 --2- 172.21.15.103:0/4267122251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d810bfc0 0x7fc4d810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.950+0000 7fc4d7e00640 1 --2- 172.21.15.103:0/4267122251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4d81034b0 0x7fc4d810b9a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.950+0000 7fc4d7e00640 1 --2- 172.21.15.103:0/4267122251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4d8102b90 0x7fc4d8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.950+0000 7fc4d7e00640 1 -- 172.21.15.103:0/4267122251 >> 172.21.15.103:0/4267122251 conn(0x7fc4d80fc610 msgr2=0x7fc4d80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:15.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.950+0000 7fc4d7e00640 1 -- 172.21.15.103:0/4267122251 shutdown_connections 2024-09-16T02:06:15.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.950+0000 7fc4d7e00640 1 -- 172.21.15.103:0/4267122251 wait complete. 2024-09-16T02:06:15.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.951+0000 7fc4d7e00640 1 Processor -- start 2024-09-16T02:06:15.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.951+0000 7fc4d7e00640 1 -- start start 2024-09-16T02:06:15.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.952+0000 7fc4d7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4d8102b90 0x7fc4d81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.952+0000 7fc4d7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d81034b0 0x7fc4d8198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.952+0000 7fc4d7e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4d810bfc0 0x7fc4d819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.952+0000 7fc4d7e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8111dc0 con 0x7fc4d8102b90 2024-09-16T02:06:15.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.952+0000 7fc4d7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8111c40 con 0x7fc4d81034b0 2024-09-16T02:06:15.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.952+0000 7fc4d7e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8111f40 con 0x7fc4d810bfc0 2024-09-16T02:06:15.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.952+0000 7fc4d7400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4d810bfc0 0x7fc4d819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:15.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.952+0000 7fc4d7400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4d810bfc0 0x7fc4d819cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:44806/0 (socket says 172.21.15.103:44806) 2024-09-16T02:06:15.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.952+0000 7fc4d7400640 1 -- 172.21.15.103:0/1700609 learned_addr learned my addr 172.21.15.103:0/1700609 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:15.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.952+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4d8102b90 0x7fc4d81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:15.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.953+0000 7fc4d7400640 1 -- 172.21.15.103:0/1700609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d81034b0 msgr2=0x7fc4d8198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:15.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.953+0000 7fc4d7400640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d81034b0 0x7fc4d8198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.953+0000 7fc4d7400640 1 -- 172.21.15.103:0/1700609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4d8102b90 msgr2=0x7fc4d81982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:15.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.953+0000 7fc4d7400640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4d8102b90 0x7fc4d81982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.953+0000 7fc4d7400640 1 -- 172.21.15.103:0/1700609 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d819d270 con 0x7fc4d810bfc0 2024-09-16T02:06:15.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.953+0000 7fc4d7400640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4d810bfc0 0x7fc4d819cb90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000eb10 tx=0x7fc4d000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:15.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.954+0000 7fc4dca00640 1 -- 172.21.15.103:0/1700609 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0004b20 con 0x7fc4d810bfc0 2024-09-16T02:06:15.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.954+0000 7fc4dca00640 1 -- 172.21.15.103:0/1700609 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d0004cc0 con 0x7fc4d810bfc0 2024-09-16T02:06:15.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.954+0000 7fc4dca00640 1 -- 172.21.15.103:0/1700609 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0010760 con 0x7fc4d810bfc0 2024-09-16T02:06:15.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.954+0000 7fc4d7e00640 1 -- 172.21.15.103:0/1700609 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d819d560 con 0x7fc4d810bfc0 2024-09-16T02:06:15.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.954+0000 7fc4d7e00640 1 -- 172.21.15.103:0/1700609 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d81a4e40 con 0x7fc4d810bfc0 2024-09-16T02:06:15.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.958+0000 7fc4e07c1740 1 Processor -- start 2024-09-16T02:06:15.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.959+0000 7fc4dca00640 1 -- 172.21.15.103:0/1700609 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fc4d0005020 con 0x7fc4d810bfc0 2024-09-16T02:06:15.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.959+0000 7fc4e07c1740 1 -- start start 2024-09-16T02:06:15.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 0x5579dd317540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 0x5579dd321e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 0x5579dd324730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5579dd2793e0 con 0x5579dd322340 2024-09-16T02:06:15.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5579dd220d80 con 0x5579dd317a80 2024-09-16T02:06:15.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5579dd2753a0 con 0x5579dd317160 2024-09-16T02:06:15.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4cfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 0x5579dd317540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:15.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4cfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 0x5579dd317540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:44810/0 (socket says 172.21.15.103:44810) 2024-09-16T02:06:15.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4cfe00640 1 -- 172.21.15.103:0/1582998866 learned_addr learned my addr 172.21.15.103:0/1582998866 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:15.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4cfe00640 1 --2- 172.21.15.103:0/1582998866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 0x5579dd324730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.960+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/1582998866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 0x5579dd321e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.961+0000 7fc4d6a00640 1 -- 172.21.15.103:0/1582998866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 msgr2=0x5579dd317540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.961+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/1582998866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 0x5579dd317540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.961+0000 7fc4d6a00640 1 -- 172.21.15.103:0/1582998866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 msgr2=0x5579dd324730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.961+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/1582998866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 0x5579dd324730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.961+0000 7fc4d6a00640 1 -- 172.21.15.103:0/1582998866 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579dd23fae0 con 0x5579dd317a80 2024-09-16T02:06:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.961+0000 7fc4dca00640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4bc0a3420 0x7fc4bc0a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.961+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/1582998866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 0x5579dd321e00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009b70 tx=0x7fc4c80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=867b70270a14b5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.961+0000 7fc4dca00640 1 -- 172.21.15.103:0/1700609 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fc4d00d00c0 con 0x7fc4d810bfc0 2024-09-16T02:06:15.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.962+0000 7fc4cfe00640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4bc0a3420 0x7fc4bc0a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:15.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.962+0000 7fc4c7e00640 1 -- 172.21.15.103:0/1582998866 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8009e60 con 0x5579dd317a80 2024-09-16T02:06:15.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.962+0000 7fc4c7e00640 1 -- 172.21.15.103:0/1582998866 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c8012930 con 0x5579dd317a80 2024-09-16T02:06:15.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.962+0000 7fc4c7e00640 1 -- 172.21.15.103:0/1582998866 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c800fbb0 con 0x5579dd317a80 2024-09-16T02:06:15.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.963+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1582998866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 msgr2=0x5579dd321e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:15.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.963+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/1582998866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 0x5579dd321e00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009b70 tx=0x7fc4c80117b0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.963+0000 7fc4cfe00640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4bc0a3420 0x7fc4bc0a58e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x5579dd314a70 tx=0x7fc4c0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:15.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.963+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1582998866 shutdown_connections 2024-09-16T02:06:15.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.963+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/1582998866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 0x5579dd324730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.963+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/1582998866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 0x5579dd321e00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.963+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/1582998866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 0x5579dd317540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.963+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1582998866 >> 172.21.15.103:0/1582998866 conn(0x5579dd3136f0 msgr2=0x5579dd313ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:15.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.963+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1582998866 shutdown_connections 2024-09-16T02:06:15.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.963+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1582998866 wait complete. 2024-09-16T02:06:15.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.964+0000 7fc4e07c1740 1 -- start start 2024-09-16T02:06:15.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.964+0000 7fc4e07c1740 1 Processor -- start 2024-09-16T02:06:15.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.964+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 0x5579dd31f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.964+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 0x5579dd31f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.965+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 0x5579dd31fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.965+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5579dd324fb0 con 0x5579dd322340 2024-09-16T02:06:15.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.965+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5579dd2340f0 con 0x5579dd317a80 2024-09-16T02:06:15.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.965+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5579dd325130 con 0x5579dd317160 2024-09-16T02:06:15.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.965+0000 7fc4e07c1740 10 client.0 _ll_register_callbacks cb 0x5579dd2259e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:15.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.965+0000 7fc4d7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 0x5579dd31fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:15.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.965+0000 7fc4d7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 0x5579dd31fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38922/0 (socket says 172.21.15.103:38922) 2024-09-16T02:06:15.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.965+0000 7fc4d7400640 1 -- 172.21.15.103:0/1336269570 learned_addr learned my addr 172.21.15.103:0/1336269570 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:15.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.965+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 0x5579dd31f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.966+0000 7fc4cfe00640 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 0x5579dd31f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.966+0000 7fc4d6a00640 1 -- 172.21.15.103:0/1336269570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 msgr2=0x5579dd31f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.966+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 0x5579dd31f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.966+0000 7fc4d6a00640 1 -- 172.21.15.103:0/1336269570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 msgr2=0x5579dd31fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.966+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 0x5579dd31fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.966+0000 7fc4d6a00640 1 -- 172.21.15.103:0/1336269570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579dcfea3c0 con 0x5579dd317160 2024-09-16T02:06:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.966+0000 7fc4d7400640 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 0x5579dd31fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.966+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 0x5579dd31f210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009b70 tx=0x7fc4c8012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5265066083789482 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.966+0000 7fc4c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:06:15.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.966+0000 7fc4c4c00640 1 -- 172.21.15.103:0/1336269570 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8028070 con 0x5579dd317160 2024-09-16T02:06:15.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.967+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1336269570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c8012930 con 0x5579dd317160 2024-09-16T02:06:15.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.967+0000 7fc4c4c00640 1 -- 172.21.15.103:0/1336269570 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fc4c800f930 con 0x5579dd317160 2024-09-16T02:06:15.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.967+0000 7fc4c4c00640 1 -- 172.21.15.103:0/1336269570 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc4c8034050 con 0x5579dd317160 2024-09-16T02:06:15.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.967+0000 7fc4e07c1740 10 client.25129 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:15.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.967+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1336269570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5579dd330c40 con 0x5579dd317160 2024-09-16T02:06:15.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.967+0000 7fc4c4c00640 1 -- 172.21.15.103:0/1336269570 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c8012930 con 0x5579dd317160 2024-09-16T02:06:15.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.968+0000 7fc4c4c00640 1 -- 172.21.15.103:0/1336269570 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc4c80046c0 con 0x5579dd317160 2024-09-16T02:06:15.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.968+0000 7fc4e07c1740 10 client.25129 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:15.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.968+0000 7fc4e07c1740 10 client.25129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:15.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.968+0000 7fc4e07c1740 20 client.25129 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:15.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.968+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5579dd332050 0x5579dd334440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:15.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.968+0000 7fc4e07c1740 4 client.25129 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:06:15.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.968+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1336269570 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579dd286950 con 0x5579dd332050 2024-09-16T02:06:15.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.968+0000 7fc4cfe00640 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5579dd332050 0x5579dd334440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:15.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.969+0000 7fc4cfe00640 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5579dd332050 0x5579dd334440 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3131c2daaf4c63d server_cookie=5751235fc292cdc in_seq=0 out_seq=0 2024-09-16T02:06:15.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.969+0000 7fc4c4c00640 10 client.25129 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:15.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.970+0000 7fc4c4c00640 1 -- 172.21.15.103:0/1336269570 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579dd286950 con 0x5579dd332050 2024-09-16T02:06:15.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.970+0000 7fc4c4c00640 10 client.25129 handle_command_reply: tid=0 2024-09-16T02:06:15.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.970+0000 7fc4e07c1740 1 client.25129 shutdown 2024-09-16T02:06:15.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.970+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1336269570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 msgr2=0x5579dd31f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:15.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.970+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 0x5579dd31f210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009b70 tx=0x7fc4c8012680 comp rx=0 tx=0).stop 2024-09-16T02:06:15.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.971+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1336269570 shutdown_connections 2024-09-16T02:06:15.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.971+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5579dd332050 0x5579dd334440 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.971+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd322340 0x5579dd31fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.971+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317a80 0x5579dd31f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.971+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/1336269570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317160 0x5579dd31f210 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:15.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.971+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1336269570 >> 172.21.15.103:0/1336269570 conn(0x5579dd3136f0 msgr2=0x5579dd3193d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:15.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.971+0000 7fc4c4c00640 0 client.25129 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:15.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.971+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1336269570 shutdown_connections 2024-09-16T02:06:15.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.971+0000 7fc4e07c1740 1 -- 172.21.15.103:0/1336269570 wait complete. 2024-09-16T02:06:15.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:15.971+0000 7fc4e07c1740 20 client.25129 trim_cache size 0 max 16384 2024-09-16T02:06:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.005+0000 7fc4e07c1740 1 Processor -- start 2024-09-16T02:06:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.005+0000 7fc4e07c1740 1 -- start start 2024-09-16T02:06:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.005+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317160 0x5579dd31eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.005+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317a80 0x5579dd320a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.005+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd322340 0x5579dd3399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.005+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81add60 con 0x5579dd317a80 2024-09-16T02:06:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.005+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81ab630 con 0x5579dd317160 2024-09-16T02:06:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.005+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81a9480 con 0x5579dd322340 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317a80 0x5579dd320a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4d6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317160 0x5579dd31eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317a80 0x5579dd320a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38934/0 (socket says 172.21.15.103:38934) 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4cfe00640 1 -- 172.21.15.103:0/3152630268 learned_addr learned my addr 172.21.15.103:0/3152630268 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4d7400640 1 --2- 172.21.15.103:0/3152630268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd322340 0x5579dd3399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4cfe00640 1 -- 172.21.15.103:0/3152630268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd322340 msgr2=0x5579dd3399e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4cfe00640 1 --2- 172.21.15.103:0/3152630268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd322340 0x5579dd3399e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4cfe00640 1 -- 172.21.15.103:0/3152630268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317160 msgr2=0x5579dd31eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4cfe00640 1 --2- 172.21.15.103:0/3152630268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317160 0x5579dd31eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4cfe00640 1 -- 172.21.15.103:0/3152630268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579dd313520 con 0x5579dd317a80 2024-09-16T02:06:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.006+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/3152630268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317160 0x5579dd31eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:16.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.007+0000 7fc4cfe00640 1 --2- 172.21.15.103:0/3152630268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317a80 0x5579dd320a70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc4c002a9a0 tx=0x7fc4c0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d6a8ebd280b305 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:16.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.007+0000 7fc4c7400640 1 -- 172.21.15.103:0/3152630268 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0035040 con 0x5579dd317a80 2024-09-16T02:06:16.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.007+0000 7fc4c7400640 1 -- 172.21.15.103:0/3152630268 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c0003d90 con 0x5579dd317a80 2024-09-16T02:06:16.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.007+0000 7fc4c7400640 1 -- 172.21.15.103:0/3152630268 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c003c470 con 0x5579dd317a80 2024-09-16T02:06:16.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.008+0000 7fc4e07c1740 1 -- 172.21.15.103:0/3152630268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317a80 msgr2=0x5579dd320a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:16.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.008+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/3152630268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317a80 0x5579dd320a70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc4c002a9a0 tx=0x7fc4c0009260 comp rx=0 tx=0).stop 2024-09-16T02:06:16.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.008+0000 7fc4e07c1740 1 -- 172.21.15.103:0/3152630268 shutdown_connections 2024-09-16T02:06:16.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.008+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/3152630268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd322340 0x5579dd3399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.008+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/3152630268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317a80 0x5579dd320a70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.008+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/3152630268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd317160 0x5579dd31eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.008+0000 7fc4e07c1740 1 -- 172.21.15.103:0/3152630268 >> 172.21.15.103:0/3152630268 conn(0x5579dd3136f0 msgr2=0x5579dd30e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:16.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.008+0000 7fc4e07c1740 1 -- 172.21.15.103:0/3152630268 shutdown_connections 2024-09-16T02:06:16.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.009+0000 7fc4e07c1740 1 -- 172.21.15.103:0/3152630268 wait complete. 2024-09-16T02:06:16.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.009+0000 7fc4e07c1740 1 -- start start 2024-09-16T02:06:16.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.009+0000 7fc4e07c1740 1 Processor -- start 2024-09-16T02:06:16.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317160 0x5579dd32aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:16.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317a80 0x5579dd31bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:16.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4e07c1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd322340 0x5579dd31c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:16.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81a9af0 con 0x5579dd317160 2024-09-16T02:06:16.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81aaf60 con 0x5579dd322340 2024-09-16T02:06:16.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4e07c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81aa1c0 con 0x5579dd317a80 2024-09-16T02:06:16.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4e07c1740 10 client.0 _ll_register_callbacks cb 0x7fc4c800f550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:16.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4d6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317160 0x5579dd32aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:16.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4cfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317a80 0x5579dd31bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:16.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4d6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317160 0x5579dd32aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38940/0 (socket says 172.21.15.103:38940) 2024-09-16T02:06:16.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4d6a00640 1 -- 172.21.15.103:0/696361943 learned_addr learned my addr 172.21.15.103:0/696361943 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:16.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.010+0000 7fc4d7400640 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd322340 0x5579dd31c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:16.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.011+0000 7fc4d7400640 1 -- 172.21.15.103:0/696361943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317a80 msgr2=0x5579dd31bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:16.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.011+0000 7fc4d7400640 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317a80 0x5579dd31bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.011+0000 7fc4d7400640 1 -- 172.21.15.103:0/696361943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317160 msgr2=0x5579dd32aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:16.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.011+0000 7fc4d7400640 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317160 0x5579dd32aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.011+0000 7fc4d7400640 1 -- 172.21.15.103:0/696361943 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579dd31cc30 con 0x5579dd322340 2024-09-16T02:06:16.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.011+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317160 0x5579dd32aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:16.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.011+0000 7fc4d7400640 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd322340 0x5579dd31c400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0098960 tx=0x7fc4d009aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ffdd67ba421d650 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:16.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.011+0000 7fc4c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:06:16.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.011+0000 7fc4c4c00640 1 -- 172.21.15.103:0/696361943 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d009c500 con 0x5579dd322340 2024-09-16T02:06:16.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.012+0000 7fc4c4c00640 1 -- 172.21.15.103:0/696361943 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fc4d00cdc40 con 0x5579dd322340 2024-09-16T02:06:16.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.012+0000 7fc4e07c1740 1 -- 172.21.15.103:0/696361943 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c0003d90 con 0x5579dd322340 2024-09-16T02:06:16.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.012+0000 7fc4c4c00640 1 -- 172.21.15.103:0/696361943 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc4d00ac050 con 0x5579dd322340 2024-09-16T02:06:16.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.012+0000 7fc4e07c1740 10 client.25151 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:16.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.012+0000 7fc4e07c1740 1 -- 172.21.15.103:0/696361943 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5579dd39ca00 con 0x5579dd322340 2024-09-16T02:06:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.013+0000 7fc4c4c00640 1 -- 172.21.15.103:0/696361943 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c0003d90 con 0x5579dd322340 2024-09-16T02:06:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.013+0000 7fc4c4c00640 1 -- 172.21.15.103:0/696361943 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc4d0005310 con 0x5579dd322340 2024-09-16T02:06:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.013+0000 7fc4e07c1740 10 client.25151 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.013+0000 7fc4e07c1740 10 client.25151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.013+0000 7fc4e07c1740 20 client.25151 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.013+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5579dd39d120 0x5579dd39f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.013+0000 7fc4e07c1740 4 client.25151 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.013+0000 7fc4e07c1740 1 -- 172.21.15.103:0/696361943 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5579dd280e60 con 0x5579dd39d120 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.013+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5579dd39d120 0x5579dd39f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.014+0000 7fc4d6a00640 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5579dd39d120 0x5579dd39f540 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60e33bfe49a0407 server_cookie=18e3099b5f8dc1e in_seq=0 out_seq=0 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.014+0000 7fc4c4c00640 10 client.25151 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.014+0000 7fc4c4c00640 1 -- 172.21.15.103:0/696361943 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5579dd280e60 con 0x5579dd39d120 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.014+0000 7fc4c4c00640 10 client.25151 handle_command_reply: tid=0 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.014+0000 7fc4e07c1740 1 client.25151 shutdown 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.014+0000 7fc4e07c1740 1 -- 172.21.15.103:0/696361943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd322340 msgr2=0x5579dd31c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.014+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd322340 0x5579dd31c400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0098960 tx=0x7fc4d009aa10 comp rx=0 tx=0).stop 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.015+0000 7fc4e07c1740 1 -- 172.21.15.103:0/696361943 shutdown_connections 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.015+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5579dd39d120 0x5579dd39f540 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.015+0000 7fc4c4c00640 0 client.25151 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:16.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.015+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579dd322340 0x5579dd31c400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.015+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5579dd317a80 0x5579dd31bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.015+0000 7fc4e07c1740 1 --2- 172.21.15.103:0/696361943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579dd317160 0x5579dd32aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.015+0000 7fc4e07c1740 1 -- 172.21.15.103:0/696361943 >> 172.21.15.103:0/696361943 conn(0x5579dd3136f0 msgr2=0x5579dd318bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:16.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.015+0000 7fc4e07c1740 1 -- 172.21.15.103:0/696361943 shutdown_connections 2024-09-16T02:06:16.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.015+0000 7fc4e07c1740 1 -- 172.21.15.103:0/696361943 wait complete. 2024-09-16T02:06:16.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.015+0000 7fc4e07c1740 20 client.25151 trim_cache size 0 max 16384 2024-09-16T02:06:16.020 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:06:16.020 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-16T02:06:16.020 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-16T02:06:16.020 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:06:16.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.016+0000 7fc4d7e00640 1 -- 172.21.15.103:0/1700609 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4bc0a3420 msgr2=0x7fc4bc0a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:16.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.016+0000 7fc4d7e00640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4bc0a3420 0x7fc4bc0a58e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x5579dd314a70 tx=0x7fc4c0000bc0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.016+0000 7fc4d7e00640 1 -- 172.21.15.103:0/1700609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4d810bfc0 msgr2=0x7fc4d819cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:16.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.016+0000 7fc4d7e00640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4d810bfc0 0x7fc4d819cb90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000eb10 tx=0x7fc4d000c2d0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.016+0000 7fc4d7e00640 1 -- 172.21.15.103:0/1700609 shutdown_connections 2024-09-16T02:06:16.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.017+0000 7fc4d7e00640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc4bc0a3420 0x7fc4bc0a58e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.017+0000 7fc4d7e00640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4d810bfc0 0x7fc4d819cb90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.017+0000 7fc4d7e00640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d81034b0 0x7fc4d8198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.017+0000 7fc4d7e00640 1 --2- 172.21.15.103:0/1700609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4d8102b90 0x7fc4d81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:16.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.017+0000 7fc4d7e00640 1 -- 172.21.15.103:0/1700609 >> 172.21.15.103:0/1700609 conn(0x7fc4d80fc610 msgr2=0x7fc4d80fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:16.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.017+0000 7fc4d7e00640 1 -- 172.21.15.103:0/1700609 shutdown_connections 2024-09-16T02:06:16.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:16.017+0000 7fc4d7e00640 1 -- 172.21.15.103:0/1700609 wait complete. 2024-09-16T02:06:16.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:15 smithi136 ceph-mon[35323]: pgmap v364: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-16T02:06:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:15 smithi172 ceph-mon[35340]: pgmap v364: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-16T02:06:16.751 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T02:06:16.752 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:06:17.253 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:17.729 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:17 smithi103 ceph-mon[30398]: pgmap v365: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T02:06:17.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.890+0000 7f3c67e00640 1 -- 172.21.15.103:0/2232956665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c68100780 msgr2=0x7f3c68100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.890+0000 7f3c67e00640 1 --2- 172.21.15.103:0/2232956665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c68100780 0x7f3c68100b60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009900 tx=0x7f3c5802a410 comp rx=0 tx=0).stop 2024-09-16T02:06:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.890+0000 7f3c67e00640 1 -- 172.21.15.103:0/2232956665 shutdown_connections 2024-09-16T02:06:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.890+0000 7f3c67e00640 1 --2- 172.21.15.103:0/2232956665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c6810e330 0x7f3c68110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.890+0000 7f3c67e00640 1 --2- 172.21.15.103:0/2232956665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c68101130 0x7f3c6810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.890+0000 7f3c67e00640 1 --2- 172.21.15.103:0/2232956665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c68100780 0x7f3c68100b60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.890+0000 7f3c67e00640 1 -- 172.21.15.103:0/2232956665 >> 172.21.15.103:0/2232956665 conn(0x7f3c680fc630 msgr2=0x7f3c680fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.891+0000 7f3c67e00640 1 -- 172.21.15.103:0/2232956665 shutdown_connections 2024-09-16T02:06:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.891+0000 7f3c67e00640 1 -- 172.21.15.103:0/2232956665 wait complete. 2024-09-16T02:06:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.892+0000 7f3c67e00640 1 Processor -- start 2024-09-16T02:06:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.892+0000 7f3c67e00640 1 -- start start 2024-09-16T02:06:17.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.892+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c68100780 0x7f3c6810ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:17.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.892+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c68101130 0x7f3c6810cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.892+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c6810e330 0x7f3c681a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.892+0000 7f3c67e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c68103cd0 con 0x7f3c68100780 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.892+0000 7f3c67e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3c68103b50 con 0x7f3c68101130 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.892+0000 7f3c67e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c68103e50 con 0x7f3c6810e330 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.893+0000 7f3c66a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c68100780 0x7f3c6810ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.893+0000 7f3c66a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c68100780 0x7f3c6810ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38952/0 (socket says 172.21.15.103:38952) 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.893+0000 7f3c66a00640 1 -- 172.21.15.103:0/3755559469 learned_addr learned my addr 172.21.15.103:0/3755559469 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.893+0000 7f3c67400640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c6810e330 0x7f3c681a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.893+0000 7f3c66a00640 1 -- 172.21.15.103:0/3755559469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c6810e330 msgr2=0x7f3c681a3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.893+0000 7f3c66a00640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c6810e330 0x7f3c681a3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.893+0000 7f3c66a00640 1 -- 172.21.15.103:0/3755559469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c68101130 msgr2=0x7f3c6810cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:17.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.893+0000 7f3c66a00640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c68101130 0x7f3c6810cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:17.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.893+0000 7f3c66a00640 1 -- 172.21.15.103:0/3755559469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c681a3960 con 0x7f3c68100780 2024-09-16T02:06:17.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.894+0000 7f3c66a00640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c68100780 0x7f3c6810ca40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3c58004770 tx=0x7f3c58003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:17.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.894+0000 7f3c5f400640 1 -- 172.21.15.103:0/3755559469 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5803f070 con 0x7f3c68100780 2024-09-16T02:06:17.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.894+0000 7f3c5f400640 1 -- 172.21.15.103:0/3755559469 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c58031ca0 con 0x7f3c68100780 2024-09-16T02:06:17.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.894+0000 7f3c5f400640 1 -- 172.21.15.103:0/3755559469 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58035030 con 0x7f3c68100780 2024-09-16T02:06:17.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.894+0000 7f3c67e00640 1 -- 172.21.15.103:0/3755559469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c681a3bf0 con 0x7f3c68100780 2024-09-16T02:06:17.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.894+0000 7f3c67e00640 1 -- 172.21.15.103:0/3755559469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c681a4080 con 0x7f3c68100780 2024-09-16T02:06:17.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.896+0000 7f3c67e00640 1 -- 172.21.15.103:0/3755559469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c68100b60 con 0x7f3c68100780 2024-09-16T02:06:17.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.899+0000 7f3c5f400640 1 -- 172.21.15.103:0/3755559469 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f3c58042070 con 0x7f3c68100780 2024-09-16T02:06:17.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.901+0000 7f3c5f400640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3c4c0a3350 0x7f3c4c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:17.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.901+0000 7f3c5f400640 1 -- 172.21.15.103:0/3755559469 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f3c580f0790 con 0x7f3c68100780 2024-09-16T02:06:17.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.901+0000 7f3c66000640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3c4c0a3350 0x7f3c4c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:17.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.902+0000 7f3c66000640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3c4c0a3350 0x7f3c4c0a5810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c6810eb30 tx=0x7f3c54006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:17.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:17.905+0000 7f3c5f400640 1 -- 172.21.15.103:0/3755559469 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c58049030 con 0x7f3c68100780 2024-09-16T02:06:18.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:17 smithi136 ceph-mon[35323]: pgmap v365: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T02:06:18.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:17 smithi172 ceph-mon[35340]: pgmap v365: 97 pgs: 97 active+clean; 663 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-16T02:06:18.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.233+0000 7f3c67e00640 1 -- 172.21.15.103:0/3755559469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c68106bb0 con 0x7f3c68100780 2024-09-16T02:06:18.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.234+0000 7f3c5f400640 1 -- 172.21.15.103:0/3755559469 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7f3c580bd270 con 0x7f3c68100780 2024-09-16T02:06:18.237 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:06:18.237 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":24,"btime":"2024-09-16T02:05:32:550231+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:05:31.661572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24506,24505]},"id":1}]} 2024-09-16T02:06:18.237 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 24 2024-09-16T02:06:18.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 -- 172.21.15.103:0/3755559469 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3c4c0a3350 msgr2=0x7f3c4c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:18.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3c4c0a3350 0x7f3c4c0a5810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c6810eb30 tx=0x7f3c54006c90 comp rx=0 tx=0).stop 2024-09-16T02:06:18.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 -- 172.21.15.103:0/3755559469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c68100780 msgr2=0x7f3c6810ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:18.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c68100780 0x7f3c6810ca40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3c58004770 tx=0x7f3c58003c10 comp rx=0 tx=0).stop 2024-09-16T02:06:18.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 -- 172.21.15.103:0/3755559469 shutdown_connections 2024-09-16T02:06:18.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f3c4c0a3350 0x7f3c4c0a5810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:18.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c6810e330 0x7f3c681a3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:18.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c68101130 0x7f3c6810cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:18.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 --2- 172.21.15.103:0/3755559469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c68100780 0x7f3c6810ca40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:18.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 -- 172.21.15.103:0/3755559469 >> 172.21.15.103:0/3755559469 conn(0x7f3c680fc630 msgr2=0x7f3c680fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:18.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 -- 172.21.15.103:0/3755559469 shutdown_connections 2024-09-16T02:06:18.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:18.237+0000 7f3c67e00640 1 -- 172.21.15.103:0/3755559469 wait complete. 2024-09-16T02:06:18.893 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:18 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3755559469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:06:18.957 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T02:06:19.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:18 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3755559469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:06:19.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:18 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3755559469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:06:19.448 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:19.922 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:19 smithi103 ceph-mon[30398]: pgmap v366: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-16T02:06:20.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:19 smithi136 ceph-mon[35323]: pgmap v366: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-16T02:06:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:19 smithi172 ceph-mon[35340]: pgmap v366: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-16T02:06:20.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.093+0000 7ff56ea00640 1 -- 172.21.15.103:0/4165199215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff56810bfc0 msgr2=0x7ff56810e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.093+0000 7ff56ea00640 1 --2- 172.21.15.103:0/4165199215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff56810bfc0 0x7ff56810e460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff560009ca0 tx=0x7ff56002a950 comp rx=0 tx=0).stop 2024-09-16T02:06:20.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.094+0000 7ff56ea00640 1 -- 172.21.15.103:0/4165199215 shutdown_connections 2024-09-16T02:06:20.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.094+0000 7ff56ea00640 1 --2- 172.21.15.103:0/4165199215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff56810bfc0 0x7ff56810e460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.094+0000 7ff56ea00640 1 --2- 172.21.15.103:0/4165199215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5681034b0 0x7ff56810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.094+0000 7ff56ea00640 1 --2- 172.21.15.103:0/4165199215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff568102b90 0x7ff568102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.094+0000 7ff56ea00640 1 -- 172.21.15.103:0/4165199215 >> 172.21.15.103:0/4165199215 conn(0x7ff5680fc610 msgr2=0x7ff5680fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.095+0000 7ff56ea00640 1 -- 172.21.15.103:0/4165199215 shutdown_connections 2024-09-16T02:06:20.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.095+0000 7ff56ea00640 1 -- 172.21.15.103:0/4165199215 wait complete. 2024-09-16T02:06:20.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.096+0000 7ff56ea00640 1 Processor -- start 2024-09-16T02:06:20.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.097+0000 7ff56ea00640 1 -- start start 2024-09-16T02:06:20.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.097+0000 7ff56ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff568102b90 0x7ff5681007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.097+0000 7ff56ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5681034b0 0x7ff568100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.097+0000 7ff56ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56810bfc0 0x7ff568101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.097+0000 7ff56ea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff568111e50 con 0x7ff56810bfc0 2024-09-16T02:06:20.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.097+0000 7ff56ea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff568111cd0 con 0x7ff568102b90 2024-09-16T02:06:20.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.097+0000 7ff56ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff568111fd0 con 0x7ff5681034b0 2024-09-16T02:06:20.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.097+0000 7ff56d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff568102b90 0x7ff5681007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.098+0000 7ff56cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5681034b0 0x7ff568100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.098+0000 7ff56d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff568102b90 0x7ff5681007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:50454/0 (socket says 172.21.15.103:50454) 2024-09-16T02:06:20.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.098+0000 7ff56d600640 1 -- 172.21.15.103:0/3317681708 learned_addr learned my addr 172.21.15.103:0/3317681708 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:20.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.098+0000 7ff56d600640 1 -- 172.21.15.103:0/3317681708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5681034b0 msgr2=0x7ff568100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.098+0000 7ff56d600640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5681034b0 0x7ff568100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.098+0000 7ff56d600640 1 -- 172.21.15.103:0/3317681708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56810bfc0 msgr2=0x7ff568101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.098+0000 7ff56d600640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56810bfc0 0x7ff568101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.098+0000 7ff56d600640 1 -- 172.21.15.103:0/3317681708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5681aac40 con 0x7ff568102b90 2024-09-16T02:06:20.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.098+0000 7ff56d600640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff568102b90 0x7ff5681007f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00b4a0 tx=0x7ff55c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.099+0000 7ff566000640 1 -- 172.21.15.103:0/3317681708 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c014070 con 0x7ff568102b90 2024-09-16T02:06:20.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.099+0000 7ff566000640 1 -- 172.21.15.103:0/3317681708 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff55c010af0 con 0x7ff568102b90 2024-09-16T02:06:20.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.099+0000 7ff56ea00640 1 -- 172.21.15.103:0/3317681708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5681aae70 con 0x7ff568102b90 2024-09-16T02:06:20.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.099+0000 7ff566000640 1 -- 172.21.15.103:0/3317681708 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c017d10 con 0x7ff568102b90 2024-09-16T02:06:20.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.099+0000 7ff56ea00640 1 -- 172.21.15.103:0/3317681708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5681ab2d0 con 0x7ff568102b90 2024-09-16T02:06:20.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.104+0000 7ff572c49740 1 Processor -- start 2024-09-16T02:06:20.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.105+0000 7ff572c49740 1 -- start start 2024-09-16T02:06:20.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.105+0000 7ff566000640 1 -- 172.21.15.103:0/3317681708 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7ff55c017070 con 0x7ff568102b90 2024-09-16T02:06:20.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.106+0000 7ff572c49740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.106+0000 7ff572c49740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916c1760 0x559c916cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.106+0000 7ff572c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 0x559c916ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.106+0000 7ff572c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559c916273e0 con 0x559c916cc020 2024-09-16T02:06:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.106+0000 7ff572c49740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559c915ced80 con 0x559c916c0e40 2024-09-16T02:06:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.106+0000 7ff572c49740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559c916233a0 con 0x559c916c1760 2024-09-16T02:06:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.107+0000 7ff56e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.107+0000 7ff56e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:50470/0 (socket says 172.21.15.103:50470) 2024-09-16T02:06:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.107+0000 7ff56e000640 1 -- 172.21.15.103:0/2250900241 learned_addr learned my addr 172.21.15.103:0/2250900241 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:20.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.108+0000 7ff56cc00640 1 --2- 172.21.15.103:0/2250900241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 0x559c916ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.108+0000 7ff566000640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff5500a32d0 0x7ff5500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.108+0000 7ff56d600640 1 --2- 172.21.15.103:0/2250900241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916c1760 0x559c916cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.108+0000 7ff56e000640 1 -- 172.21.15.103:0/2250900241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916c1760 msgr2=0x559c916cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.108+0000 7ff56e000640 1 --2- 172.21.15.103:0/2250900241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916c1760 0x559c916cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.108+0000 7ff56e000640 1 -- 172.21.15.103:0/2250900241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 msgr2=0x559c916ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.108+0000 7ff566000640 1 -- 172.21.15.103:0/3317681708 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7ff55c0cef10 con 0x7ff568102b90 2024-09-16T02:06:20.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.109+0000 7ff56e000640 1 --2- 172.21.15.103:0/2250900241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 0x559c916ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.109+0000 7ff56e000640 1 -- 172.21.15.103:0/2250900241 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c915edae0 con 0x559c916c0e40 2024-09-16T02:06:20.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.109+0000 7ff56e000640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff5500a32d0 0x7ff5500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.109+0000 7ff56e000640 1 --2- 172.21.15.103:0/2250900241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c1220 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff560009dc0 tx=0x7ff56002f5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ca010ddeefe91b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.110+0000 7ff55a600640 1 -- 172.21.15.103:0/2250900241 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560034040 con 0x559c916c0e40 2024-09-16T02:06:20.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.110+0000 7ff55a600640 1 -- 172.21.15.103:0/2250900241 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff560004040 con 0x559c916c0e40 2024-09-16T02:06:20.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.110+0000 7ff55a600640 1 -- 172.21.15.103:0/2250900241 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560039650 con 0x559c916c0e40 2024-09-16T02:06:20.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.111+0000 7ff572c49740 1 -- 172.21.15.103:0/2250900241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 msgr2=0x559c916c1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.111+0000 7ff56e000640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff5500a32d0 0x7ff5500a5790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff56004a000 tx=0x7ff56004b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.111+0000 7ff572c49740 1 --2- 172.21.15.103:0/2250900241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c1220 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff560009dc0 tx=0x7ff56002f5d0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.111+0000 7ff572c49740 1 -- 172.21.15.103:0/2250900241 shutdown_connections 2024-09-16T02:06:20.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.111+0000 7ff572c49740 1 --2- 172.21.15.103:0/2250900241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 0x559c916ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.111+0000 7ff572c49740 1 --2- 172.21.15.103:0/2250900241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916c1760 0x559c916cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.111+0000 7ff572c49740 1 --2- 172.21.15.103:0/2250900241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c1220 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.111+0000 7ff572c49740 1 -- 172.21.15.103:0/2250900241 >> 172.21.15.103:0/2250900241 conn(0x559c916bd3d0 msgr2=0x559c916bd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.112+0000 7ff572c49740 1 -- 172.21.15.103:0/2250900241 shutdown_connections 2024-09-16T02:06:20.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.112+0000 7ff572c49740 1 -- 172.21.15.103:0/2250900241 wait complete. 2024-09-16T02:06:20.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.112+0000 7ff572c49740 1 -- start start 2024-09-16T02:06:20.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.113+0000 7ff572c49740 1 Processor -- start 2024-09-16T02:06:20.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.113+0000 7ff572c49740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.113+0000 7ff572c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916c1760 0x559c916c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.113+0000 7ff572c49740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916c9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.113+0000 7ff572c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559c916cebf0 con 0x559c916c1760 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.113+0000 7ff572c49740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559c915e20f0 con 0x559c916c0e40 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.113+0000 7ff572c49740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559c916ced70 con 0x559c916cc020 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.114+0000 7ff56d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.114+0000 7ff56d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:50474/0 (socket says 172.21.15.103:50474) 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.114+0000 7ff56d600640 1 -- 172.21.15.103:0/3211783222 learned_addr learned my addr 172.21.15.103:0/3211783222 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.114+0000 7ff572c49740 10 client.0 _ll_register_callbacks cb 0x559c915c9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.114+0000 7ff56e000640 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.114+0000 7ff56d600640 1 -- 172.21.15.103:0/3211783222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 msgr2=0x559c916c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.114+0000 7ff56d600640 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.114+0000 7ff56d600640 1 -- 172.21.15.103:0/3211783222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916c1760 msgr2=0x559c916c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.114+0000 7ff56cc00640 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916c1760 0x559c916c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.115+0000 7ff56d600640 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916c1760 0x559c916c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.115+0000 7ff56d600640 1 -- 172.21.15.103:0/3211783222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c913983c0 con 0x559c916c0e40 2024-09-16T02:06:20.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.115+0000 7ff56e000640 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:06:20.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.115+0000 7ff56cc00640 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916c1760 0x559c916c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:06:20.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.116+0000 7ff56d600640 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c8ef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00be60 tx=0x7ff55c009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5572135192dda84d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.116+0000 7ff54ea00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:06:20.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.116+0000 7ff54ea00640 1 -- 172.21.15.103:0/3211783222 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c01e070 con 0x559c916c0e40 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.116+0000 7ff54ea00640 1 -- 172.21.15.103:0/3211783222 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7ff55c017360 con 0x559c916c0e40 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.117+0000 7ff572c49740 1 -- 172.21.15.103:0/3211783222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff560004040 con 0x559c916c0e40 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.117+0000 7ff54ea00640 1 -- 172.21.15.103:0/3211783222 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff55c09f810 con 0x559c916c0e40 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.117+0000 7ff572c49740 10 client.25163 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.117+0000 7ff572c49740 1 -- 172.21.15.103:0/3211783222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559c916dac70 con 0x559c916c0e40 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.117+0000 7ff54ea00640 1 -- 172.21.15.103:0/3211783222 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff560004040 con 0x559c916c0e40 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.117+0000 7ff54ea00640 1 -- 172.21.15.103:0/3211783222 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff55c097ad0 con 0x559c916c0e40 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.118+0000 7ff572c49740 10 client.25163 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.118+0000 7ff572c49740 10 client.25163 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.118+0000 7ff572c49740 20 client.25163 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.118+0000 7ff572c49740 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x559c916dc130 0x559c916de520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.118+0000 7ff572c49740 4 client.25163 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:06:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.118+0000 7ff572c49740 1 -- 172.21.15.103:0/3211783222 --> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c91634950 con 0x559c916dc130 2024-09-16T02:06:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.119+0000 7ff56cc00640 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x559c916dc130 0x559c916de520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.120+0000 7ff56cc00640 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x559c916dc130 0x559c916de520 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6dd153bfdc9a6b3 server_cookie=53b64eeefd187a8d in_seq=0 out_seq=0 2024-09-16T02:06:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.120+0000 7ff54ea00640 10 client.25163 ms_handle_connect on v2:172.21.15.172:6838/3474994563 2024-09-16T02:06:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.120+0000 7ff54ea00640 1 -- 172.21.15.103:0/3211783222 <== mds.2 v2:172.21.15.172:6838/3474994563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c91634950 con 0x559c916dc130 2024-09-16T02:06:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.120+0000 7ff54ea00640 10 client.25163 handle_command_reply: tid=0 2024-09-16T02:06:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.121+0000 7ff572c49740 1 client.25163 shutdown 2024-09-16T02:06:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.121+0000 7ff572c49740 1 -- 172.21.15.103:0/3211783222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 msgr2=0x559c916c8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.121+0000 7ff572c49740 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c8ef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00be60 tx=0x7ff55c009360 comp rx=0 tx=0).stop 2024-09-16T02:06:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.121+0000 7ff572c49740 1 -- 172.21.15.103:0/3211783222 shutdown_connections 2024-09-16T02:06:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.121+0000 7ff572c49740 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x559c916dc130 0x559c916de520 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.121+0000 7ff572c49740 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.121+0000 7ff572c49740 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916c1760 0x559c916c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.121+0000 7ff572c49740 1 --2- 172.21.15.103:0/3211783222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c8ef0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.121+0000 7ff572c49740 1 -- 172.21.15.103:0/3211783222 >> 172.21.15.103:0/3211783222 conn(0x559c916bd3d0 msgr2=0x559c916c30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.122+0000 7ff54ea00640 0 client.25163 ms_handle_reset on v2:172.21.15.172:6838/3474994563 2024-09-16T02:06:20.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.122+0000 7ff572c49740 1 -- 172.21.15.103:0/3211783222 shutdown_connections 2024-09-16T02:06:20.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.122+0000 7ff572c49740 1 -- 172.21.15.103:0/3211783222 wait complete. 2024-09-16T02:06:20.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.122+0000 7ff572c49740 20 client.25163 trim_cache size 0 max 16384 2024-09-16T02:06:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.149+0000 7ff572c49740 1 Processor -- start 2024-09-16T02:06:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.149+0000 7ff572c49740 1 -- start start 2024-09-16T02:06:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.149+0000 7ff572c49740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.149+0000 7ff572c49740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916c1760 0x559c916e3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.149+0000 7ff572c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 0x559c916e4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.149+0000 7ff572c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5681b66c0 con 0x559c916cc020 2024-09-16T02:06:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.149+0000 7ff572c49740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5681b3f90 con 0x559c916c0e40 2024-09-16T02:06:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.149+0000 7ff572c49740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff5681b1de0 con 0x559c916c1760 2024-09-16T02:06:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.150+0000 7ff56cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916c1760 0x559c916e3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.150+0000 7ff56d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.150+0000 7ff56d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:50480/0 (socket says 172.21.15.103:50480) 2024-09-16T02:06:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.150+0000 7ff56d600640 1 -- 172.21.15.103:0/3483916710 learned_addr learned my addr 172.21.15.103:0/3483916710 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.150+0000 7ff56d600640 1 -- 172.21.15.103:0/3483916710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916c1760 msgr2=0x559c916e3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.150+0000 7ff56d600640 1 --2- 172.21.15.103:0/3483916710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916c1760 0x559c916e3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.150+0000 7ff56d600640 1 -- 172.21.15.103:0/3483916710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 msgr2=0x559c916e4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.151+0000 7ff56e000640 1 --2- 172.21.15.103:0/3483916710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 0x559c916e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.151+0000 7ff56d600640 1 --2- 172.21.15.103:0/3483916710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 0x559c916e4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.151+0000 7ff56d600640 1 -- 172.21.15.103:0/3483916710 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c916c7460 con 0x559c916c0e40 2024-09-16T02:06:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.151+0000 7ff56e000640 1 --2- 172.21.15.103:0/3483916710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 0x559c916e4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:06:20.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.152+0000 7ff56d600640 1 --2- 172.21.15.103:0/3483916710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c2fa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff55c09b3f0 tx=0x7ff55c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd78a916036af87e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.152+0000 7ff559c00640 1 -- 172.21.15.103:0/3483916710 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c01e020 con 0x559c916c0e40 2024-09-16T02:06:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.152+0000 7ff559c00640 1 -- 172.21.15.103:0/3483916710 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff55c0cce50 con 0x559c916c0e40 2024-09-16T02:06:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.152+0000 7ff559c00640 1 -- 172.21.15.103:0/3483916710 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c097ad0 con 0x559c916c0e40 2024-09-16T02:06:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.153+0000 7ff572c49740 1 -- 172.21.15.103:0/3483916710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 msgr2=0x559c916c2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.153+0000 7ff572c49740 1 --2- 172.21.15.103:0/3483916710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c2fa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff55c09b3f0 tx=0x7ff55c0046c0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.154+0000 7ff572c49740 1 -- 172.21.15.103:0/3483916710 shutdown_connections 2024-09-16T02:06:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.154+0000 7ff572c49740 1 --2- 172.21.15.103:0/3483916710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916cc020 0x559c916e4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.154+0000 7ff572c49740 1 --2- 172.21.15.103:0/3483916710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916c1760 0x559c916e3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.154+0000 7ff572c49740 1 --2- 172.21.15.103:0/3483916710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c0e40 0x559c916c2fa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.154+0000 7ff572c49740 1 -- 172.21.15.103:0/3483916710 >> 172.21.15.103:0/3483916710 conn(0x559c916bd3d0 msgr2=0x559c916c2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.154+0000 7ff572c49740 1 -- 172.21.15.103:0/3483916710 shutdown_connections 2024-09-16T02:06:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.154+0000 7ff572c49740 1 -- 172.21.15.103:0/3483916710 wait complete. 2024-09-16T02:06:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.154+0000 7ff572c49740 1 -- start start 2024-09-16T02:06:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.155+0000 7ff572c49740 1 Processor -- start 2024-09-16T02:06:20.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.155+0000 7ff572c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916c0e40 0x559c916d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.155+0000 7ff572c49740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c1760 0x559c916d3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.155+0000 7ff572c49740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916d3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.155+0000 7ff572c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5681b2450 con 0x559c916c0e40 2024-09-16T02:06:20.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.155+0000 7ff572c49740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5681b38c0 con 0x559c916c1760 2024-09-16T02:06:20.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.155+0000 7ff572c49740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff5681b2b20 con 0x559c916cc020 2024-09-16T02:06:20.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.155+0000 7ff572c49740 10 client.0 _ll_register_callbacks cb 0x7ff55c0092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:20.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.156+0000 7ff56e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916d3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.156+0000 7ff56e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916d3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:34734/0 (socket says 172.21.15.103:34734) 2024-09-16T02:06:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.156+0000 7ff56e000640 1 -- 172.21.15.103:0/1918549387 learned_addr learned my addr 172.21.15.103:0/1918549387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.156+0000 7ff56cc00640 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c1760 0x559c916d3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.157+0000 7ff56cc00640 1 -- 172.21.15.103:0/1918549387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 msgr2=0x559c916d3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.157+0000 7ff56cc00640 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916d3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.157+0000 7ff56cc00640 1 -- 172.21.15.103:0/1918549387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916c0e40 msgr2=0x559c916d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.157+0000 7ff56cc00640 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916c0e40 0x559c916d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.157+0000 7ff56cc00640 1 -- 172.21.15.103:0/1918549387 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c916d4240 con 0x559c916c1760 2024-09-16T02:06:20.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.157+0000 7ff56e000640 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916d3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.157+0000 7ff56cc00640 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c1760 0x559c916d3500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x559c916be750 tx=0x7ff55401a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b0b94c9feb1f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.157+0000 7ff54ea00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:06:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.158+0000 7ff54ea00640 1 -- 172.21.15.103:0/1918549387 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554023640 con 0x559c916c1760 2024-09-16T02:06:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.158+0000 7ff54ea00640 1 -- 172.21.15.103:0/1918549387 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7ff554012440 con 0x559c916c1760 2024-09-16T02:06:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.158+0000 7ff572c49740 1 -- 172.21.15.103:0/1918549387 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff55c0cce50 con 0x559c916c1760 2024-09-16T02:06:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.158+0000 7ff54ea00640 1 -- 172.21.15.103:0/1918549387 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff5540237e0 con 0x559c916c1760 2024-09-16T02:06:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.158+0000 7ff572c49740 10 client.25169 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.158+0000 7ff572c49740 1 -- 172.21.15.103:0/1918549387 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559c91758400 con 0x559c916c1760 2024-09-16T02:06:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.159+0000 7ff54ea00640 1 -- 172.21.15.103:0/1918549387 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff55c0cce50 con 0x559c916c1760 2024-09-16T02:06:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.159+0000 7ff54ea00640 1 -- 172.21.15.103:0/1918549387 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff554021070 con 0x559c916c1760 2024-09-16T02:06:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.159+0000 7ff572c49740 10 client.25169 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.159+0000 7ff572c49740 10 client.25169 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T02:06:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.159+0000 7ff572c49740 20 client.25169 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.159+0000 7ff572c49740 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x559c91758b80 0x559c9175afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.159+0000 7ff572c49740 4 client.25169 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:06:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.159+0000 7ff572c49740 1 -- 172.21.15.103:0/1918549387 --> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c9162ee60 con 0x559c91758b80 2024-09-16T02:06:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.160+0000 7ff56d600640 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x559c91758b80 0x559c9175afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.160+0000 7ff56d600640 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x559c91758b80 0x559c9175afa0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df1489e87e854c05 server_cookie=61bcbb1d544f99e1 in_seq=0 out_seq=0 2024-09-16T02:06:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.160+0000 7ff54ea00640 10 client.25169 ms_handle_connect on v2:172.21.15.172:6838/3474994563 2024-09-16T02:06:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff54ea00640 1 -- 172.21.15.103:0/1918549387 <== mds.2 v2:172.21.15.172:6838/3474994563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c9162ee60 con 0x559c91758b80 2024-09-16T02:06:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff54ea00640 10 client.25169 handle_command_reply: tid=0 2024-09-16T02:06:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 client.25169 shutdown 2024-09-16T02:06:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 -- 172.21.15.103:0/1918549387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c1760 msgr2=0x559c916d3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c1760 0x559c916d3500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x559c916be750 tx=0x7ff55401a660 comp rx=0 tx=0).stop 2024-09-16T02:06:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 -- 172.21.15.103:0/1918549387 shutdown_connections 2024-09-16T02:06:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x559c91758b80 0x559c9175afa0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c916cc020 0x559c916d3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c916c1760 0x559c916d3500 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 --2- 172.21.15.103:0/1918549387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c916c0e40 0x559c916d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 -- 172.21.15.103:0/1918549387 >> 172.21.15.103:0/1918549387 conn(0x559c916bd3d0 msgr2=0x559c916c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff54ea00640 0 client.25169 ms_handle_reset on v2:172.21.15.172:6838/3474994563 2024-09-16T02:06:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 -- 172.21.15.103:0/1918549387 shutdown_connections 2024-09-16T02:06:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.161+0000 7ff572c49740 1 -- 172.21.15.103:0/1918549387 wait complete. 2024-09-16T02:06:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.162+0000 7ff572c49740 20 client.25169 trim_cache size 0 max 16384 2024-09-16T02:06:20.165 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-16T02:06:20.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 -- 172.21.15.103:0/3317681708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff5500a32d0 msgr2=0x7ff5500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff5500a32d0 0x7ff5500a5790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff56004a000 tx=0x7ff56004b000 comp rx=0 tx=0).stop 2024-09-16T02:06:20.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 -- 172.21.15.103:0/3317681708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff568102b90 msgr2=0x7ff5681007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff568102b90 0x7ff5681007f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00b4a0 tx=0x7ff55c00b960 comp rx=0 tx=0).stop 2024-09-16T02:06:20.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 -- 172.21.15.103:0/3317681708 shutdown_connections 2024-09-16T02:06:20.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7ff5500a32d0 0x7ff5500a5790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56810bfc0 0x7ff568101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5681034b0 0x7ff568100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 --2- 172.21.15.103:0/3317681708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff568102b90 0x7ff5681007f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 -- 172.21.15.103:0/3317681708 >> 172.21.15.103:0/3317681708 conn(0x7ff5680fc610 msgr2=0x7ff5680fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 -- 172.21.15.103:0/3317681708 shutdown_connections 2024-09-16T02:06:20.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:20.163+0000 7ff56ea00640 1 -- 172.21.15.103:0/3317681708 wait complete. 2024-09-16T02:06:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.819+0000 7f69fe400640 1 -- 172.21.15.103:0/3536853916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f8101490 msgr2=0x7f69f8101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.819+0000 7f69fe400640 1 --2- 172.21.15.103:0/3536853916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f8101490 0x7f69f8101940 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009ca0 tx=0x7f69f002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.820+0000 7f69fe400640 1 -- 172.21.15.103:0/3536853916 shutdown_connections 2024-09-16T02:06:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.820+0000 7f69fe400640 1 --2- 172.21.15.103:0/3536853916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f8101490 0x7f69f8101940 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.820+0000 7f69fe400640 1 --2- 172.21.15.103:0/3536853916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f8100af0 0x7f69f8100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.820+0000 7f69fe400640 1 --2- 172.21.15.103:0/3536853916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f81001b0 0x7f69f81005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.820+0000 7f69fe400640 1 -- 172.21.15.103:0/3536853916 >> 172.21.15.103:0/3536853916 conn(0x7f69f80fbfa0 msgr2=0x7f69f80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.820+0000 7f69fe400640 1 -- 172.21.15.103:0/3536853916 shutdown_connections 2024-09-16T02:06:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.820+0000 7f69fe400640 1 -- 172.21.15.103:0/3536853916 wait complete. 2024-09-16T02:06:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.821+0000 7f69fe400640 1 Processor -- start 2024-09-16T02:06:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.821+0000 7f69fe400640 1 -- start start 2024-09-16T02:06:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.821+0000 7f69fe400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81001b0 0x7f69f819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.821+0000 7f69fe400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f8100af0 0x7f69f819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69fe400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f8101490 0x7f69f81a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69fe400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810f7e0 con 0x7f69f81001b0 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69fe400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810f660 con 0x7f69f8100af0 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69fe400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810f960 con 0x7f69f8101490 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69fd000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81001b0 0x7f69f819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69fd000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81001b0 0x7f69f819e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47600/0 (socket says 172.21.15.103:47600) 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69fd000640 1 -- 172.21.15.103:0/2528598596 learned_addr learned my addr 172.21.15.103:0/2528598596 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69fda00640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f8101490 0x7f69f81a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69f7e00640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f8100af0 0x7f69f819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69f7e00640 1 -- 172.21.15.103:0/2528598596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f8101490 msgr2=0x7f69f81a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69f7e00640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f8101490 0x7f69f81a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69f7e00640 1 -- 172.21.15.103:0/2528598596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81001b0 msgr2=0x7f69f819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69f7e00640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81001b0 0x7f69f819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.822+0000 7f69f7e00640 1 -- 172.21.15.103:0/2528598596 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f81a5e40 con 0x7f69f8100af0 2024-09-16T02:06:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.823+0000 7f69f7e00640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f8100af0 0x7f69f819e760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f69e400d6d0 tx=0x7f69e400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.823+0000 7f69f5600640 1 -- 172.21.15.103:0/2528598596 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4017070 con 0x7f69f8100af0 2024-09-16T02:06:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.823+0000 7f69f5600640 1 -- 172.21.15.103:0/2528598596 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e4009820 con 0x7f69f8100af0 2024-09-16T02:06:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.823+0000 7f69f5600640 1 -- 172.21.15.103:0/2528598596 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4004d00 con 0x7f69f8100af0 2024-09-16T02:06:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.823+0000 7f69fe400640 1 -- 172.21.15.103:0/2528598596 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f81a6130 con 0x7f69f8100af0 2024-09-16T02:06:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.823+0000 7f69fe400640 1 -- 172.21.15.103:0/2528598596 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f81a6630 con 0x7f69f8100af0 2024-09-16T02:06:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.828+0000 7f6a027e2740 1 Processor -- start 2024-09-16T02:06:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.828+0000 7f6a027e2740 1 -- start start 2024-09-16T02:06:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.828+0000 7f69f5600640 1 -- 172.21.15.103:0/2528598596 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f69e40099d0 con 0x7f69f8100af0 2024-09-16T02:06:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.828+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53b260 0x55ae5e53b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.828+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 0x55ae5e540660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.828+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 0x55ae5e5430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.828+0000 7f6a027e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae5e4a0100 con 0x55ae5e53b260 2024-09-16T02:06:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.828+0000 7f6a027e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ae5e4481b0 con 0x55ae5e53bba0 2024-09-16T02:06:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.828+0000 7f6a027e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ae5e49c0c0 con 0x55ae5e540c30 2024-09-16T02:06:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.829+0000 7f69fd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 0x55ae5e540660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.829+0000 7f69f7e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 0x55ae5e5430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.829+0000 7f69fda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53b260 0x55ae5e53b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.829+0000 7f69fd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 0x55ae5e540660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:50520/0 (socket says 172.21.15.103:50520) 2024-09-16T02:06:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.829+0000 7f69fd000640 1 -- 172.21.15.103:0/1283539160 learned_addr learned my addr 172.21.15.103:0/1283539160 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.829+0000 7f69f7e00640 1 -- 172.21.15.103:0/1283539160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 msgr2=0x55ae5e540660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.829+0000 7f69f7e00640 1 --2- 172.21.15.103:0/1283539160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 0x55ae5e540660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.829+0000 7f69f7e00640 1 -- 172.21.15.103:0/1283539160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53b260 msgr2=0x55ae5e53b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.829+0000 7f69f7e00640 1 --2- 172.21.15.103:0/1283539160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53b260 0x55ae5e53b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.829+0000 7f69f7e00640 1 -- 172.21.15.103:0/1283539160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae5e1b3210 con 0x55ae5e540c30 2024-09-16T02:06:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.830+0000 7f69fda00640 1 --2- 172.21.15.103:0/1283539160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53b260 0x55ae5e53b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.830+0000 7f69fd000640 1 --2- 172.21.15.103:0/1283539160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 0x55ae5e540660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.830+0000 7f69f7e00640 1 --2- 172.21.15.103:0/1283539160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 0x55ae5e5430f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f69e40ccef0 tx=0x7f69e40d5b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31bf9c93ee83b51c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.830+0000 7f69ea000640 1 -- 172.21.15.103:0/1283539160 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e40e2070 con 0x55ae5e540c30 2024-09-16T02:06:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.830+0000 7f69f5600640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f69e00a3350 0x7f69e00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.830+0000 7f69ea000640 1 -- 172.21.15.103:0/1283539160 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e40cfbd0 con 0x55ae5e540c30 2024-09-16T02:06:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.830+0000 7f69ea000640 1 -- 172.21.15.103:0/1283539160 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e40d15b0 con 0x55ae5e540c30 2024-09-16T02:06:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.831+0000 7f69f5600640 1 -- 172.21.15.103:0/2528598596 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f69e40cf620 con 0x7f69f8100af0 2024-09-16T02:06:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.831+0000 7f6a027e2740 1 -- 172.21.15.103:0/1283539160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 msgr2=0x55ae5e5430f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.831+0000 7f6a027e2740 1 --2- 172.21.15.103:0/1283539160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 0x55ae5e5430f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f69e40ccef0 tx=0x7f69e40d5b80 comp rx=0 tx=0).stop 2024-09-16T02:06:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.831+0000 7f69fda00640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f69e00a3350 0x7f69e00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.832+0000 7f6a027e2740 1 -- 172.21.15.103:0/1283539160 shutdown_connections 2024-09-16T02:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.832+0000 7f6a027e2740 1 --2- 172.21.15.103:0/1283539160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 0x55ae5e5430f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.832+0000 7f6a027e2740 1 --2- 172.21.15.103:0/1283539160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 0x55ae5e540660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.832+0000 7f6a027e2740 1 --2- 172.21.15.103:0/1283539160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53b260 0x55ae5e53b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.832+0000 7f6a027e2740 1 -- 172.21.15.103:0/1283539160 >> 172.21.15.103:0/1283539160 conn(0x55ae5e5368d0 msgr2=0x55ae5e536cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.833+0000 7f69fda00640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f69e00a3350 0x7f69e00a5810 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f69f0005e30 tx=0x7f69f0031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.833+0000 7f6a027e2740 1 -- 172.21.15.103:0/1283539160 shutdown_connections 2024-09-16T02:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.833+0000 7f6a027e2740 1 -- 172.21.15.103:0/1283539160 wait complete. 2024-09-16T02:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.833+0000 7f6a027e2740 1 -- start start 2024-09-16T02:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.834+0000 7f6a027e2740 1 Processor -- start 2024-09-16T02:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.834+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 0x55ae5e556480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.834+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53bba0 0x55ae5e5569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.834+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 0x55ae5e55da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.834+0000 7f6a027e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae5e554c10 con 0x55ae5e540c30 2024-09-16T02:06:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.834+0000 7f6a027e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ae5e429090 con 0x55ae5e53b260 2024-09-16T02:06:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.834+0000 7f6a027e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ae5e554d90 con 0x55ae5e53bba0 2024-09-16T02:06:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.834+0000 7f6a027e2740 10 client.0 _ll_register_callbacks cb 0x55ae5e47df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69fd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 0x55ae5e556480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69fd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 0x55ae5e556480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:50524/0 (socket says 172.21.15.103:50524) 2024-09-16T02:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69fd000640 1 -- 172.21.15.103:0/2164900547 learned_addr learned my addr 172.21.15.103:0/2164900547 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69fda00640 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 0x55ae5e55da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69f7e00640 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53bba0 0x55ae5e5569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69f7e00640 1 -- 172.21.15.103:0/2164900547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 msgr2=0x55ae5e556480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69f7e00640 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 0x55ae5e556480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69f7e00640 1 -- 172.21.15.103:0/2164900547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 msgr2=0x55ae5e55da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69f7e00640 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 0x55ae5e55da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69f7e00640 1 -- 172.21.15.103:0/2164900547 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae5e27f620 con 0x55ae5e53bba0 2024-09-16T02:06:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.835+0000 7f69fd000640 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 0x55ae5e556480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.836+0000 7f69f7e00640 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53bba0 0x55ae5e5569c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f69e409ba40 tx=0x7f69e40057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ad5ab7a189c3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.836+0000 7f69dea00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:06:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.836+0000 7f69dea00640 1 -- 172.21.15.103:0/2164900547 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4020070 con 0x55ae5e53bba0 2024-09-16T02:06:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.836+0000 7f69dea00640 1 -- 172.21.15.103:0/2164900547 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f69e40cf9d0 con 0x55ae5e53bba0 2024-09-16T02:06:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.836+0000 7f6a027e2740 1 -- 172.21.15.103:0/2164900547 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69e40cfbd0 con 0x55ae5e53bba0 2024-09-16T02:06:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.836+0000 7f69dea00640 1 -- 172.21.15.103:0/2164900547 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f69e4097d50 con 0x55ae5e53bba0 2024-09-16T02:06:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.837+0000 7f6a027e2740 10 client.25165 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.837+0000 7f6a027e2740 1 -- 172.21.15.103:0/2164900547 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ae5e554c10 con 0x55ae5e53bba0 2024-09-16T02:06:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.837+0000 7f69dea00640 1 -- 172.21.15.103:0/2164900547 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e40cfbd0 con 0x55ae5e53bba0 2024-09-16T02:06:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.837+0000 7f69dea00640 1 -- 172.21.15.103:0/2164900547 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f69e4098120 con 0x55ae5e53bba0 2024-09-16T02:06:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.837+0000 7f6a027e2740 10 client.25165 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.837+0000 7f6a027e2740 10 client.25165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.837+0000 7f6a027e2740 20 client.25165 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.837+0000 7f6a027e2740 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55ae5e561970 0x55ae5e563d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.838+0000 7f6a027e2740 4 client.25165 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:06:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.838+0000 7f6a027e2740 1 -- 172.21.15.103:0/2164900547 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae5e4ad830 con 0x55ae5e561970 2024-09-16T02:06:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.838+0000 7f69fd000640 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55ae5e561970 0x55ae5e563d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.838+0000 7f69fd000640 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55ae5e561970 0x55ae5e563d60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d7498730df027b server_cookie=d9718a59f4a17b8c in_seq=0 out_seq=0 2024-09-16T02:06:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.839+0000 7f69dea00640 10 client.25165 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.839+0000 7f69dea00640 1 -- 172.21.15.103:0/2164900547 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae5e4ad830 con 0x55ae5e561970 2024-09-16T02:06:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.839+0000 7f69dea00640 10 client.25165 handle_command_reply: tid=0 2024-09-16T02:06:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.839+0000 7f6a027e2740 1 client.25165 shutdown 2024-09-16T02:06:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 1 -- 172.21.15.103:0/2164900547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53bba0 msgr2=0x55ae5e5569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53bba0 0x55ae5e5569c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f69e409ba40 tx=0x7f69e40057f0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 1 -- 172.21.15.103:0/2164900547 shutdown_connections 2024-09-16T02:06:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55ae5e561970 0x55ae5e563d60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 0x55ae5e55da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53bba0 0x55ae5e5569c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 1 --2- 172.21.15.103:0/2164900547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 0x55ae5e556480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 1 -- 172.21.15.103:0/2164900547 >> 172.21.15.103:0/2164900547 conn(0x55ae5e5368d0 msgr2=0x55ae5e5380e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f69dea00640 0 client.25165 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 1 -- 172.21.15.103:0/2164900547 shutdown_connections 2024-09-16T02:06:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 1 -- 172.21.15.103:0/2164900547 wait complete. 2024-09-16T02:06:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.840+0000 7f6a027e2740 20 client.25165 trim_cache size 0 max 16384 2024-09-16T02:06:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.869+0000 7f6a027e2740 1 Processor -- start 2024-09-16T02:06:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.869+0000 7f6a027e2740 1 -- start start 2024-09-16T02:06:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.869+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 0x55ae5e536cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.869+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53bba0 0x55ae5e55d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.869+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 0x55ae5e5371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.869+0000 7f6a027e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81a9870 con 0x55ae5e53bba0 2024-09-16T02:06:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.869+0000 7f6a027e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81a7140 con 0x55ae5e53b260 2024-09-16T02:06:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.869+0000 7f6a027e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8072b20 con 0x55ae5e540c30 2024-09-16T02:06:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.870+0000 7f69f7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53bba0 0x55ae5e55d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.870+0000 7f69f7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53bba0 0x55ae5e55d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47626/0 (socket says 172.21.15.103:47626) 2024-09-16T02:06:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.870+0000 7f69f7e00640 1 -- 172.21.15.103:0/1675220759 learned_addr learned my addr 172.21.15.103:0/1675220759 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.870+0000 7f69fda00640 1 --2- 172.21.15.103:0/1675220759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 0x55ae5e5371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.870+0000 7f69fd000640 1 --2- 172.21.15.103:0/1675220759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 0x55ae5e536cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.870+0000 7f69fda00640 1 -- 172.21.15.103:0/1675220759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 msgr2=0x55ae5e536cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.870+0000 7f69fda00640 1 --2- 172.21.15.103:0/1675220759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 0x55ae5e536cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.870+0000 7f69fda00640 1 -- 172.21.15.103:0/1675220759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53bba0 msgr2=0x55ae5e55d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.870+0000 7f69fda00640 1 --2- 172.21.15.103:0/1675220759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53bba0 0x55ae5e55d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.870+0000 7f69fda00640 1 -- 172.21.15.103:0/1675220759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae5e5366f0 con 0x55ae5e540c30 2024-09-16T02:06:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.871+0000 7f69fda00640 1 --2- 172.21.15.103:0/1675220759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 0x55ae5e5371f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f69f002ade0 tx=0x7f69f0041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3ae202caaf49aa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.871+0000 7f69e9600640 1 -- 172.21.15.103:0/1675220759 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0003af0 con 0x55ae5e540c30 2024-09-16T02:06:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.871+0000 7f69e9600640 1 -- 172.21.15.103:0/1675220759 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f0036050 con 0x55ae5e540c30 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.871+0000 7f69e9600640 1 -- 172.21.15.103:0/1675220759 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f002f610 con 0x55ae5e540c30 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.872+0000 7f6a027e2740 1 -- 172.21.15.103:0/1675220759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 msgr2=0x55ae5e5371f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.872+0000 7f6a027e2740 1 --2- 172.21.15.103:0/1675220759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 0x55ae5e5371f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f69f002ade0 tx=0x7f69f0041000 comp rx=0 tx=0).stop 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.872+0000 7f6a027e2740 1 -- 172.21.15.103:0/1675220759 shutdown_connections 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.872+0000 7f6a027e2740 1 --2- 172.21.15.103:0/1675220759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e540c30 0x55ae5e5371f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.872+0000 7f6a027e2740 1 --2- 172.21.15.103:0/1675220759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e53bba0 0x55ae5e55d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.872+0000 7f6a027e2740 1 --2- 172.21.15.103:0/1675220759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53b260 0x55ae5e536cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.872+0000 7f6a027e2740 1 -- 172.21.15.103:0/1675220759 >> 172.21.15.103:0/1675220759 conn(0x55ae5e5368d0 msgr2=0x55ae5e556480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.872+0000 7f6a027e2740 1 -- 172.21.15.103:0/1675220759 shutdown_connections 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.872+0000 7f6a027e2740 1 -- 172.21.15.103:0/1675220759 wait complete. 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.872+0000 7f6a027e2740 1 -- start start 2024-09-16T02:06:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.873+0000 7f6a027e2740 1 Processor -- start 2024-09-16T02:06:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.873+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53b260 0x55ae5e55a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.873+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 0x55ae5e55ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.873+0000 7f6a027e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 0x55ae5e5ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.873+0000 7f6a027e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8073190 con 0x55ae5e540c30 2024-09-16T02:06:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.873+0000 7f6a027e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81a6a70 con 0x55ae5e53bba0 2024-09-16T02:06:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.873+0000 7f6a027e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8073860 con 0x55ae5e53b260 2024-09-16T02:06:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.873+0000 7f6a027e2740 10 client.0 _ll_register_callbacks cb 0x7f69e40e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.873+0000 7f69f7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 0x55ae5e55ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 0x55ae5e5ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fd000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53b260 0x55ae5e55a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 0x55ae5e5ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47632/0 (socket says 172.21.15.103:47632) 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fda00640 1 -- 172.21.15.103:0/134788646 learned_addr learned my addr 172.21.15.103:0/134788646 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fd000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53b260 0x55ae5e55a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:34778/0 (socket says 172.21.15.103:34778) 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fd000640 1 -- 172.21.15.103:0/134788646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 msgr2=0x55ae5e55ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fd000640 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 0x55ae5e55ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fd000640 1 -- 172.21.15.103:0/134788646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 msgr2=0x55ae5e5ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fd000640 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 0x55ae5e5ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fd000640 1 -- 172.21.15.103:0/134788646 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae5e55b420 con 0x55ae5e53b260 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fda00640 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 0x55ae5e5ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69fd000640 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53b260 0x55ae5e55a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55ae5e5644d0 tx=0x7f69ec0039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ecb8ffa71862555 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.874+0000 7f69dea00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.875+0000 7f69dea00640 1 -- 172.21.15.103:0/134788646 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec005220 con 0x55ae5e53b260 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.875+0000 7f6a027e2740 1 -- 172.21.15.103:0/134788646 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f0036050 con 0x55ae5e53b260 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.875+0000 7f69dea00640 1 -- 172.21.15.103:0/134788646 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f69ec0053c0 con 0x55ae5e53b260 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.875+0000 7f69dea00640 1 -- 172.21.15.103:0/134788646 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f69ec0055c0 con 0x55ae5e53b260 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.875+0000 7f6a027e2740 10 client.25171 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.875+0000 7f6a027e2740 1 -- 172.21.15.103:0/134788646 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ae5e5cbc10 con 0x55ae5e53b260 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.875+0000 7f69dea00640 1 -- 172.21.15.103:0/134788646 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f0036050 con 0x55ae5e53b260 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.876+0000 7f69dea00640 1 -- 172.21.15.103:0/134788646 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f69ec030960 con 0x55ae5e53b260 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.876+0000 7f6a027e2740 10 client.25171 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.876+0000 7f6a027e2740 10 client.25171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.876+0000 7f6a027e2740 20 client.25171 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.876+0000 7f6a027e2740 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55ae5e5cc510 0x55ae5e5ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.876+0000 7f6a027e2740 4 client.25171 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T02:06:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.876+0000 7f6a027e2740 1 -- 172.21.15.103:0/134788646 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae5e4a7b80 con 0x55ae5e5cc510 2024-09-16T02:06:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.877+0000 7f69f7e00640 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55ae5e5cc510 0x55ae5e5ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.877+0000 7f69dea00640 10 client.25171 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.877+0000 7f69f7e00640 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55ae5e5cc510 0x55ae5e5ce900 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d659ce7aeaa20cea server_cookie=886c4c405a5521fa in_seq=0 out_seq=0 2024-09-16T02:06:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.885+0000 7f69dea00640 1 -- 172.21.15.103:0/134788646 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae5e4a7b80 con 0x55ae5e5cc510 2024-09-16T02:06:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.885+0000 7f69dea00640 10 client.25171 handle_command_reply: tid=0 2024-09-16T02:06:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.885+0000 7f6a027e2740 1 client.25171 shutdown 2024-09-16T02:06:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 1 -- 172.21.15.103:0/134788646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53b260 msgr2=0x55ae5e55a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53b260 0x55ae5e55a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55ae5e5644d0 tx=0x7f69ec0039c0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 1 -- 172.21.15.103:0/134788646 shutdown_connections 2024-09-16T02:06:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55ae5e5cc510 0x55ae5e5ce900 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5e540c30 0x55ae5e5ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ae5e53bba0 0x55ae5e55ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 1 --2- 172.21.15.103:0/134788646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae5e53b260 0x55ae5e55a840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 1 -- 172.21.15.103:0/134788646 >> 172.21.15.103:0/134788646 conn(0x55ae5e5368d0 msgr2=0x55ae5e5351f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f69dea00640 0 client.25171 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 1 -- 172.21.15.103:0/134788646 shutdown_connections 2024-09-16T02:06:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 1 -- 172.21.15.103:0/134788646 wait complete. 2024-09-16T02:06:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.886+0000 7f6a027e2740 20 client.25171 trim_cache size 0 max 16384 2024-09-16T02:06:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-16T02:06:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-16T02:06:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-16T02:06:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-16T02:06:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 -- 172.21.15.103:0/2528598596 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f69e00a3350 msgr2=0x7f69e00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f69e00a3350 0x7f69e00a5810 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f69f0005e30 tx=0x7f69f0031bb0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 -- 172.21.15.103:0/2528598596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f8100af0 msgr2=0x7f69f819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f8100af0 0x7f69f819e760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f69e400d6d0 tx=0x7f69e400db90 comp rx=0 tx=0).stop 2024-09-16T02:06:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 -- 172.21.15.103:0/2528598596 shutdown_connections 2024-09-16T02:06:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f69e00a3350 0x7f69e00a5810 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f8101490 0x7f69f81a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f8100af0 0x7f69f819e760 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 --2- 172.21.15.103:0/2528598596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81001b0 0x7f69f819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 -- 172.21.15.103:0/2528598596 >> 172.21.15.103:0/2528598596 conn(0x7f69f80fbfa0 msgr2=0x7f69f80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 -- 172.21.15.103:0/2528598596 shutdown_connections 2024-09-16T02:06:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:20.887+0000 7f69fe400640 1 -- 172.21.15.103:0/2528598596 wait complete. 2024-09-16T02:06:20.984 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T02:06:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:21.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:21 smithi103 ceph-mon[30398]: pgmap v367: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T02:06:22.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:21 smithi136 ceph-mon[35323]: pgmap v367: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T02:06:22.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:21 smithi172 ceph-mon[35340]: pgmap v367: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T02:06:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.188+0000 7f2cdda00640 1 -- 172.21.15.103:0/33997023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cd8102b90 msgr2=0x7f2cd8102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.188+0000 7f2cdda00640 1 --2- 172.21.15.103:0/33997023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cd8102b90 0x7f2cd8102f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009860 tx=0x7f2cc802a3d0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.188+0000 7f2cdda00640 1 -- 172.21.15.103:0/33997023 shutdown_connections 2024-09-16T02:06:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.188+0000 7f2cdda00640 1 --2- 172.21.15.103:0/33997023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cd810bfc0 0x7f2cd810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.188+0000 7f2cdda00640 1 --2- 172.21.15.103:0/33997023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cd81034b0 0x7f2cd810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.188+0000 7f2cdda00640 1 --2- 172.21.15.103:0/33997023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cd8102b90 0x7f2cd8102f70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.188+0000 7f2cdda00640 1 -- 172.21.15.103:0/33997023 >> 172.21.15.103:0/33997023 conn(0x7f2cd80fc610 msgr2=0x7f2cd80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.189+0000 7f2cdda00640 1 -- 172.21.15.103:0/33997023 shutdown_connections 2024-09-16T02:06:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.189+0000 7f2cdda00640 1 -- 172.21.15.103:0/33997023 wait complete. 2024-09-16T02:06:22.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.190+0000 7f2cdda00640 1 Processor -- start 2024-09-16T02:06:22.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.190+0000 7f2cdda00640 1 -- start start 2024-09-16T02:06:22.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.190+0000 7f2cdda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cd8102b90 0x7f2cd8103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.191+0000 7f2cdda00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cd81034b0 0x7f2cd8103f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.191+0000 7f2cdda00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cd810bfc0 0x7f2cd8104490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.191+0000 7f2cdda00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8111e10 con 0x7f2cd8102b90 2024-09-16T02:06:22.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.191+0000 7f2cdda00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8111c90 con 0x7f2cd810bfc0 2024-09-16T02:06:22.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.191+0000 7f2cdda00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8111f90 con 0x7f2cd81034b0 2024-09-16T02:06:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.191+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cd8102b90 0x7f2cd8103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.191+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cd8102b90 0x7f2cd8103a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47650/0 (socket says 172.21.15.103:47650) 2024-09-16T02:06:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.191+0000 7f2cd7e00640 1 -- 172.21.15.103:0/1636343110 learned_addr learned my addr 172.21.15.103:0/1636343110 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.191+0000 7f2cd7400640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cd81034b0 0x7f2cd8103f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.191+0000 7f2cdd000640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cd810bfc0 0x7f2cd8104490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.192+0000 7f2cd7e00640 1 -- 172.21.15.103:0/1636343110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cd81034b0 msgr2=0x7f2cd8103f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.192+0000 7f2cd7e00640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cd81034b0 0x7f2cd8103f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.192+0000 7f2cd7e00640 1 -- 172.21.15.103:0/1636343110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cd810bfc0 msgr2=0x7f2cd8104490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.192+0000 7f2cd7e00640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cd810bfc0 0x7f2cd8104490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.192+0000 7f2cd7e00640 1 -- 172.21.15.103:0/1636343110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd81aabb0 con 0x7f2cd8102b90 2024-09-16T02:06:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.192+0000 7f2cd7e00640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cd8102b90 0x7f2cd8103a10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80309a0 tx=0x7f2cc80309d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.192+0000 7f2cd4c00640 1 -- 172.21.15.103:0/1636343110 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8040070 con 0x7f2cd8102b90 2024-09-16T02:06:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.193+0000 7f2cdda00640 1 -- 172.21.15.103:0/1636343110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd81aae40 con 0x7f2cd8102b90 2024-09-16T02:06:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.193+0000 7f2cdda00640 1 -- 172.21.15.103:0/1636343110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd81ab2a0 con 0x7f2cd8102b90 2024-09-16T02:06:22.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.193+0000 7f2cd4c00640 1 -- 172.21.15.103:0/1636343110 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc8005620 con 0x7f2cd8102b90 2024-09-16T02:06:22.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.193+0000 7f2cd4c00640 1 -- 172.21.15.103:0/1636343110 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80035f0 con 0x7f2cd8102b90 2024-09-16T02:06:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.197+0000 7f2ce1bd9740 1 Processor -- start 2024-09-16T02:06:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.197+0000 7f2ce1bd9740 1 -- start start 2024-09-16T02:06:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.197+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4337fe40 0x55ca43380220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.197+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 0x55ca4338aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.197+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca4338d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.197+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca432e63e0 con 0x55ca4338b020 2024-09-16T02:06:22.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.197+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ca4328dd80 con 0x55ca4337fe40 2024-09-16T02:06:22.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.197+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ca432e23a0 con 0x55ca43380760 2024-09-16T02:06:22.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.198+0000 7f2cdd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4337fe40 0x55ca43380220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.198+0000 7f2cdd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4337fe40 0x55ca43380220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:50570/0 (socket says 172.21.15.103:50570) 2024-09-16T02:06:22.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.198+0000 7f2cdd000640 1 -- 172.21.15.103:0/3959910785 learned_addr learned my addr 172.21.15.103:0/3959910785 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:22.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.198+0000 7f2cd7400640 1 --2- 172.21.15.103:0/3959910785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca4338d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.198+0000 7f2cdd000640 1 -- 172.21.15.103:0/3959910785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 msgr2=0x55ca4338aae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T02:06:22.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.198+0000 7f2cdd000640 1 --2- 172.21.15.103:0/3959910785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 0x55ca4338aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.198+0000 7f2cdd000640 1 -- 172.21.15.103:0/3959910785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 msgr2=0x55ca4338d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:22.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.198+0000 7f2cdd000640 1 --2- 172.21.15.103:0/3959910785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca4338d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.198+0000 7f2cdd000640 1 -- 172.21.15.103:0/3959910785 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca432acae0 con 0x55ca4337fe40 2024-09-16T02:06:22.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.198+0000 7f2cd7400640 1 --2- 172.21.15.103:0/3959910785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca4338d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:06:22.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.199+0000 7f2cd4c00640 1 -- 172.21.15.103:0/1636343110 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f2cc8043070 con 0x7f2cd8102b90 2024-09-16T02:06:22.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.199+0000 7f2cdd000640 1 --2- 172.21.15.103:0/3959910785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4337fe40 0x55ca43380220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00099f0 tx=0x7f2cd00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73c869ee86256973 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.199+0000 7f2ccd600640 1 -- 172.21.15.103:0/3959910785 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd001d070 con 0x55ca4337fe40 2024-09-16T02:06:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.199+0000 7f2ccd600640 1 -- 172.21.15.103:0/3959910785 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd0003e40 con 0x55ca4337fe40 2024-09-16T02:06:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.199+0000 7f2ccd600640 1 -- 172.21.15.103:0/3959910785 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd000db80 con 0x55ca4337fe40 2024-09-16T02:06:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.200+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3959910785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4337fe40 msgr2=0x55ca43380220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.200+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/3959910785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4337fe40 0x55ca43380220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00099f0 tx=0x7f2cd00095c0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.200+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3959910785 shutdown_connections 2024-09-16T02:06:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.200+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/3959910785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca4338d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.200+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/3959910785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 0x55ca4338aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.200+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/3959910785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4337fe40 0x55ca43380220 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.200+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3959910785 >> 172.21.15.103:0/3959910785 conn(0x55ca4337c3d0 msgr2=0x55ca4337c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.200+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3959910785 shutdown_connections 2024-09-16T02:06:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.200+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3959910785 wait complete. 2024-09-16T02:06:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.201+0000 7f2cd4c00640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2cbc0a32e0 0x7f2cbc0a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.201+0000 7f2cd4c00640 1 -- 172.21.15.103:0/1636343110 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f2cc80f0690 con 0x7f2cd8102b90 2024-09-16T02:06:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.201+0000 7f2ce1bd9740 1 -- start start 2024-09-16T02:06:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.201+0000 7f2cd7400640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2cbc0a32e0 0x7f2cbc0a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.201+0000 7f2ce1bd9740 1 Processor -- start 2024-09-16T02:06:22.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.202+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433897b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.202+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 0x55ca43389cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.202+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 0x55ca4338a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.202+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca4338dbf0 con 0x55ca4337fe40 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.202+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ca432a10f0 con 0x55ca4338b020 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.202+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ca4338dd70 con 0x55ca43380760 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.202+0000 7f2ce1bd9740 10 client.0 _ll_register_callbacks cb 0x55ca43288bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.202+0000 7f2cd7400640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2cbc0a32e0 0x7f2cbc0a57a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55ca4337d750 tx=0x7f2cc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433897b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433897b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47668/0 (socket says 172.21.15.103:47668) 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cd7e00640 1 -- 172.21.15.103:0/637149779 learned_addr learned my addr 172.21.15.103:0/637149779 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cd7400640 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 0x55ca43389cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cdd000640 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 0x55ca4338a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cd7e00640 1 -- 172.21.15.103:0/637149779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 msgr2=0x55ca43389cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cd7e00640 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 0x55ca43389cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cd7e00640 1 -- 172.21.15.103:0/637149779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 msgr2=0x55ca4338a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cd7e00640 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 0x55ca4338a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cd7e00640 1 -- 172.21.15.103:0/637149779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca430573c0 con 0x55ca4337fe40 2024-09-16T02:06:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.203+0000 7f2cdd000640 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 0x55ca4338a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:06:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.204+0000 7f2cd7e00640 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433897b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009980 tx=0x7f2cc80bc720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef83bd4aa5f59eda server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.204+0000 7f2cc2000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:06:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.204+0000 7f2cc2000640 1 -- 172.21.15.103:0/637149779 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc804a040 con 0x55ca4337fe40 2024-09-16T02:06:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.204+0000 7f2cc2000640 1 -- 172.21.15.103:0/637149779 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f2cc80c5450 con 0x55ca4337fe40 2024-09-16T02:06:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.204+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/637149779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca432f3950 con 0x55ca4337fe40 2024-09-16T02:06:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.205+0000 7f2cc2000640 1 -- 172.21.15.103:0/637149779 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2cc80c0e20 con 0x55ca4337fe40 2024-09-16T02:06:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.205+0000 7f2ce1bd9740 10 client.15249 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.205+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/637149779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ca43399c40 con 0x55ca4337fe40 2024-09-16T02:06:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.205+0000 7f2cc2000640 1 -- 172.21.15.103:0/637149779 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ca432f3950 con 0x55ca4337fe40 2024-09-16T02:06:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.205+0000 7f2cc2000640 1 -- 172.21.15.103:0/637149779 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2cc80046c0 con 0x55ca4337fe40 2024-09-16T02:06:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.206+0000 7f2ce1bd9740 10 client.15249 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.206+0000 7f2ce1bd9740 10 client.15249 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T02:06:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.206+0000 7f2ce1bd9740 20 client.15249 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.206+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55ca4339b030 0x55ca4339d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.206+0000 7f2ce1bd9740 4 client.15249 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:06:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.206+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/637149779 --> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca432ede60 con 0x55ca4339b030 2024-09-16T02:06:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.207+0000 7f2cd7400640 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55ca4339b030 0x55ca4339d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.207+0000 7f2cd7400640 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55ca4339b030 0x55ca4339d450 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76aef79c574a6ab9 server_cookie=9ec9e652b73b7b6a in_seq=0 out_seq=0 2024-09-16T02:06:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.207+0000 7f2cc2000640 10 client.15249 ms_handle_connect on v2:172.21.15.136:6836/236571207 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.208+0000 7f2cc2000640 1 -- 172.21.15.103:0/637149779 <== mds.1 v2:172.21.15.136:6836/236571207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca432ede60 con 0x55ca4339b030 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.208+0000 7f2cc2000640 10 client.15249 handle_command_reply: tid=0 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.208+0000 7f2ce1bd9740 1 client.15249 shutdown 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.208+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/637149779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 msgr2=0x55ca433897b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.208+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433897b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009980 tx=0x7f2cc80bc720 comp rx=0 tx=0).stop 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.209+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/637149779 shutdown_connections 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.209+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55ca4339b030 0x55ca4339d450 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.209+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 0x55ca4338a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.209+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 0x55ca43389cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.209+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/637149779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433897b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.209+0000 7f2cc2000640 0 client.15249 ms_handle_reset on v2:172.21.15.136:6836/236571207 2024-09-16T02:06:22.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.209+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/637149779 >> 172.21.15.103:0/637149779 conn(0x55ca4337c3d0 msgr2=0x55ca433820b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:22.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.209+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/637149779 shutdown_connections 2024-09-16T02:06:22.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.209+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/637149779 wait complete. 2024-09-16T02:06:22.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.210+0000 7f2ce1bd9740 20 client.15249 trim_cache size 0 max 16384 2024-09-16T02:06:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.223+0000 7f2ce1bd9740 1 Processor -- start 2024-09-16T02:06:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.223+0000 7f2ce1bd9740 1 -- start start 2024-09-16T02:06:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.224+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca4337fe40 0x55ca433828a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.224+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca43380760 0x55ca43388820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.224+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca433a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.224+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb8010ea0 con 0x55ca4338b020 2024-09-16T02:06:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.224+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb800e770 con 0x55ca43380760 2024-09-16T02:06:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.224+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb800c5c0 con 0x55ca4337fe40 2024-09-16T02:06:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.224+0000 7f2cdd000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca433a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.224+0000 7f2cdd000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca433a2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47682/0 (socket says 172.21.15.103:47682) 2024-09-16T02:06:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.224+0000 7f2cdd000640 1 -- 172.21.15.103:0/1062762476 learned_addr learned my addr 172.21.15.103:0/1062762476 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.225+0000 7f2cd7e00640 1 --2- 172.21.15.103:0/1062762476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca4337fe40 0x55ca433828a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.225+0000 7f2cd7400640 1 --2- 172.21.15.103:0/1062762476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca43380760 0x55ca43388820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.225+0000 7f2cdd000640 1 -- 172.21.15.103:0/1062762476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca4337fe40 msgr2=0x55ca433828a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.225+0000 7f2cdd000640 1 --2- 172.21.15.103:0/1062762476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca4337fe40 0x55ca433828a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.225+0000 7f2cdd000640 1 -- 172.21.15.103:0/1062762476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca43380760 msgr2=0x55ca43388820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.225+0000 7f2cdd000640 1 --2- 172.21.15.103:0/1062762476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca43380760 0x55ca43388820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.225+0000 7f2cdd000640 1 -- 172.21.15.103:0/1062762476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4339dad0 con 0x55ca4338b020 2024-09-16T02:06:22.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.225+0000 7f2cdd000640 1 --2- 172.21.15.103:0/1062762476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca433a2d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2cd001a000 tx=0x7f2cd00097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fca08bd72e0d7b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:22.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.226+0000 7f2cccc00640 1 -- 172.21.15.103:0/1062762476 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0028070 con 0x55ca4338b020 2024-09-16T02:06:22.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.226+0000 7f2cccc00640 1 -- 172.21.15.103:0/1062762476 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd00055a0 con 0x55ca4338b020 2024-09-16T02:06:22.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.226+0000 7f2cccc00640 1 -- 172.21.15.103:0/1062762476 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd001d030 con 0x55ca4338b020 2024-09-16T02:06:22.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.227+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/1062762476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 msgr2=0x55ca433a2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:22.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.227+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/1062762476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca433a2d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2cd001a000 tx=0x7f2cd00097e0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.227+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/1062762476 shutdown_connections 2024-09-16T02:06:22.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.227+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/1062762476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4338b020 0x55ca433a2d60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.227+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/1062762476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca43380760 0x55ca43388820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.227+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/1062762476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca4337fe40 0x55ca433828a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.227+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/1062762476 >> 172.21.15.103:0/1062762476 conn(0x55ca4337c3d0 msgr2=0x55ca43389fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:22.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.227+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/1062762476 shutdown_connections 2024-09-16T02:06:22.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.227+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/1062762476 wait complete. 2024-09-16T02:06:22.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.228+0000 7f2ce1bd9740 1 -- start start 2024-09-16T02:06:22.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.228+0000 7f2ce1bd9740 1 Processor -- start 2024-09-16T02:06:22.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 0x55ca4338e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2ce1bd9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 0x55ca4338ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb800cc30 con 0x55ca4337fe40 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb800e0a0 con 0x55ca4338b020 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2ce1bd9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb800d300 con 0x55ca43380760 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2ce1bd9740 10 client.0 _ll_register_callbacks cb 0x7f2cc80097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433935d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47688/0 (socket says 172.21.15.103:47688) 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2cd7e00640 1 -- 172.21.15.103:0/3304371852 learned_addr learned my addr 172.21.15.103:0/3304371852 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.229+0000 7f2cdd000640 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 0x55ca4338ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.230+0000 7f2cd7e00640 1 -- 172.21.15.103:0/3304371852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 msgr2=0x55ca4338e900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.230+0000 7f2cd7e00640 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 0x55ca4338e900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.230+0000 7f2cd7e00640 1 -- 172.21.15.103:0/3304371852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 msgr2=0x55ca4338ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.230+0000 7f2cd7e00640 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 0x55ca4338ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.230+0000 7f2cd7e00640 1 -- 172.21.15.103:0/3304371852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4338f480 con 0x55ca4337fe40 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.230+0000 7f2cd7e00640 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433935d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80b94f0 tx=0x7f2cc80049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2186bcf523a4a382 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.230+0000 7f2cc2000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.230+0000 7f2cc2000640 1 -- 172.21.15.103:0/3304371852 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80a1060 con 0x55ca4337fe40 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.230+0000 7f2cc2000640 1 -- 172.21.15.103:0/3304371852 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f2cc80c5450 con 0x55ca4337fe40 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.230+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3304371852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd00055a0 con 0x55ca4337fe40 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.231+0000 7f2cc2000640 1 -- 172.21.15.103:0/3304371852 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2cc804a050 con 0x55ca4337fe40 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.231+0000 7f2ce1bd9740 10 client.15255 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.231+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3304371852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ca4338f480 con 0x55ca4337fe40 2024-09-16T02:06:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.231+0000 7f2cc2000640 1 -- 172.21.15.103:0/3304371852 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd00055a0 con 0x55ca4337fe40 2024-09-16T02:06:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.231+0000 7f2cc2000640 1 -- 172.21.15.103:0/3304371852 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2cc80f0110 con 0x55ca4337fe40 2024-09-16T02:06:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.231+0000 7f2ce1bd9740 10 client.15255 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.231+0000 7f2ce1bd9740 10 client.15255 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T02:06:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.231+0000 7f2ce1bd9740 20 client.15255 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.231+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55ca43405f20 0x55ca43408330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.232+0000 7f2ce1bd9740 4 client.15255 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.232+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3304371852 --> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca4326a010 con 0x55ca43405f20 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.232+0000 7f2cd7400640 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55ca43405f20 0x55ca43408330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2cc2000640 10 client.15255 ms_handle_connect on v2:172.21.15.136:6836/236571207 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2cd7400640 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55ca43405f20 0x55ca43408330 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=241b336b5b2d4643 server_cookie=fb7252742f445b1 in_seq=0 out_seq=0 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2cc2000640 1 -- 172.21.15.103:0/3304371852 <== mds.1 v2:172.21.15.136:6836/236571207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca4326a010 con 0x55ca43405f20 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2cc2000640 10 client.15255 handle_command_reply: tid=0 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2ce1bd9740 1 client.15255 shutdown 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3304371852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 msgr2=0x55ca433935d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433935d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80b94f0 tx=0x7f2cc80049d0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3304371852 shutdown_connections 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x55ca43405f20 0x55ca43408330 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca4338b020 0x55ca4338ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca43380760 0x55ca4338e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2ce1bd9740 1 --2- 172.21.15.103:0/3304371852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4337fe40 0x55ca433935d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3304371852 >> 172.21.15.103:0/3304371852 conn(0x55ca4337c3d0 msgr2=0x55ca4337ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:22.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.233+0000 7f2cc2000640 0 client.15255 ms_handle_reset on v2:172.21.15.136:6836/236571207 2024-09-16T02:06:22.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.234+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3304371852 shutdown_connections 2024-09-16T02:06:22.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.234+0000 7f2ce1bd9740 1 -- 172.21.15.103:0/3304371852 wait complete. 2024-09-16T02:06:22.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.234+0000 7f2ce1bd9740 20 client.15255 trim_cache size 0 max 16384 2024-09-16T02:06:22.238 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-16T02:06:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.235+0000 7f2cdda00640 1 -- 172.21.15.103:0/1636343110 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2cbc0a32e0 msgr2=0x7f2cbc0a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.235+0000 7f2cdda00640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2cbc0a32e0 0x7f2cbc0a57a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55ca4337d750 tx=0x7f2cc4005f90 comp rx=0 tx=0).stop 2024-09-16T02:06:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.235+0000 7f2cdda00640 1 -- 172.21.15.103:0/1636343110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cd8102b90 msgr2=0x7f2cd8103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.235+0000 7f2cdda00640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cd8102b90 0x7f2cd8103a10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80309a0 tx=0x7f2cc80309d0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.235+0000 7f2cdda00640 1 -- 172.21.15.103:0/1636343110 shutdown_connections 2024-09-16T02:06:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.235+0000 7f2cdda00640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2cbc0a32e0 0x7f2cbc0a57a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.235+0000 7f2cdda00640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cd810bfc0 0x7f2cd8104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.235+0000 7f2cdda00640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cd81034b0 0x7f2cd8103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.235+0000 7f2cdda00640 1 --2- 172.21.15.103:0/1636343110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cd8102b90 0x7f2cd8103a10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:22.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.235+0000 7f2cdda00640 1 -- 172.21.15.103:0/1636343110 >> 172.21.15.103:0/1636343110 conn(0x7f2cd80fc610 msgr2=0x7f2cd80fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:22.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.236+0000 7f2cdda00640 1 -- 172.21.15.103:0/1636343110 shutdown_connections 2024-09-16T02:06:22.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:22.236+0000 7f2cdda00640 1 -- 172.21.15.103:0/1636343110 wait complete. 2024-09-16T02:06:23.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T02:06:23.559 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:23.869 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:23 smithi103 ceph-mon[30398]: pgmap v368: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T02:06:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:23 smithi136 ceph-mon[35323]: pgmap v368: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T02:06:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:23 smithi172 ceph-mon[35340]: pgmap v368: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-16T02:06:24.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.139+0000 7f26de400640 1 -- 172.21.15.103:0/1868541247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26d8108f60 msgr2=0x7f26d8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:24.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.139+0000 7f26de400640 1 --2- 172.21.15.103:0/1868541247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26d8108f60 0x7f26d8110ae0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009be0 tx=0x7f26d002a6c0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.140+0000 7f26de400640 1 -- 172.21.15.103:0/1868541247 shutdown_connections 2024-09-16T02:06:24.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.140+0000 7f26de400640 1 --2- 172.21.15.103:0/1868541247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26d8108f60 0x7f26d8110ae0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.140+0000 7f26de400640 1 --2- 172.21.15.103:0/1868541247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d81047e0 0x7f26d8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.140+0000 7f26de400640 1 --2- 172.21.15.103:0/1868541247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8103e30 0x7f26d8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.140+0000 7f26de400640 1 -- 172.21.15.103:0/1868541247 >> 172.21.15.103:0/1868541247 conn(0x7f26d80ff910 msgr2=0x7f26d8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:24.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.140+0000 7f26de400640 1 -- 172.21.15.103:0/1868541247 shutdown_connections 2024-09-16T02:06:24.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.140+0000 7f26de400640 1 -- 172.21.15.103:0/1868541247 wait complete. 2024-09-16T02:06:24.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.141+0000 7f26de400640 1 Processor -- start 2024-09-16T02:06:24.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.142+0000 7f26de400640 1 -- start start 2024-09-16T02:06:24.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.142+0000 7f26de400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8103e30 0x7f26d80739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.142+0000 7f26de400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d81047e0 0x7f26d8073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.142+0000 7f26de400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26d8108f60 0x7f26d806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.142+0000 7f26de400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26d8113fc0 con 0x7f26d8103e30 2024-09-16T02:06:24.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.142+0000 7f26de400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f26d8113e40 con 0x7f26d8108f60 2024-09-16T02:06:24.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.142+0000 7f26de400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26d8114140 con 0x7f26d81047e0 2024-09-16T02:06:24.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.143+0000 7f26dd000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8103e30 0x7f26d80739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.143+0000 7f26dd000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8103e30 0x7f26d80739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47698/0 (socket says 172.21.15.103:47698) 2024-09-16T02:06:24.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.143+0000 7f26dd000640 1 -- 172.21.15.103:0/3045327122 learned_addr learned my addr 172.21.15.103:0/3045327122 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:24.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.143+0000 7f26d7e00640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d81047e0 0x7f26d8073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.143+0000 7f26dda00640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26d8108f60 0x7f26d806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.144+0000 7f26d7e00640 1 -- 172.21.15.103:0/3045327122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26d8108f60 msgr2=0x7f26d806cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:24.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.144+0000 7f26d7e00640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26d8108f60 0x7f26d806cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.144+0000 7f26d7e00640 1 -- 172.21.15.103:0/3045327122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8103e30 msgr2=0x7f26d80739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:24.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.144+0000 7f26d7e00640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8103e30 0x7f26d80739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.144+0000 7f26d7e00640 1 -- 172.21.15.103:0/3045327122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d806d4d0 con 0x7f26d81047e0 2024-09-16T02:06:24.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.145+0000 7f26d7e00640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d81047e0 0x7f26d8073ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26c400e9f0 tx=0x7f26c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:24.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.145+0000 7f26d5600640 1 -- 172.21.15.103:0/3045327122 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c400ccc0 con 0x7f26d81047e0 2024-09-16T02:06:24.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.145+0000 7f26de400640 1 -- 172.21.15.103:0/3045327122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d806d7c0 con 0x7f26d81047e0 2024-09-16T02:06:24.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.145+0000 7f26de400640 1 -- 172.21.15.103:0/3045327122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d80717e0 con 0x7f26d81047e0 2024-09-16T02:06:24.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.146+0000 7f26d5600640 1 -- 172.21.15.103:0/3045327122 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c4010910 con 0x7f26d81047e0 2024-09-16T02:06:24.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.146+0000 7f26d5600640 1 -- 172.21.15.103:0/3045327122 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40188a0 con 0x7f26d81047e0 2024-09-16T02:06:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.150+0000 7f26e248c740 1 Processor -- start 2024-09-16T02:06:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.150+0000 7f26e248c740 1 -- start start 2024-09-16T02:06:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.150+0000 7f26e248c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.150+0000 7f26e248c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dba0760 0x562e6dbaaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.151+0000 7f26dda00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.151+0000 7f26dda00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:34860/0 (socket says 172.21.15.103:34860) 2024-09-16T02:06:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.151+0000 7f26e248c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dbab020 0x562e6dbad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.151+0000 7f26e248c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562e6db063e0 con 0x562e6dba0760 2024-09-16T02:06:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.151+0000 7f26e248c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562e6daadd80 con 0x562e6dbab020 2024-09-16T02:06:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.151+0000 7f26e248c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562e6db023a0 con 0x562e6db9fe40 2024-09-16T02:06:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.151+0000 7f26dda00640 1 -- 172.21.15.103:0/4043366577 learned_addr learned my addr 172.21.15.103:0/4043366577 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.151+0000 7f26dd000640 1 --2- 172.21.15.103:0/4043366577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dba0760 0x562e6dbaaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.152+0000 7f26dda00640 1 -- 172.21.15.103:0/4043366577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dbab020 msgr2=0x562e6dbad410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T02:06:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.152+0000 7f26dda00640 1 --2- 172.21.15.103:0/4043366577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dbab020 0x562e6dbad410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.152+0000 7f26dda00640 1 -- 172.21.15.103:0/4043366577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dba0760 msgr2=0x562e6dbaaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:24.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.152+0000 7f26dda00640 1 --2- 172.21.15.103:0/4043366577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dba0760 0x562e6dbaaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.152+0000 7f26dda00640 1 -- 172.21.15.103:0/4043366577 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6daccae0 con 0x562e6db9fe40 2024-09-16T02:06:24.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.152+0000 7f26dd000640 1 --2- 172.21.15.103:0/4043366577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dba0760 0x562e6dbaaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:06:24.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.152+0000 7f26dda00640 1 --2- 172.21.15.103:0/4043366577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba0220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f26d002acb0 tx=0x7f26d0030a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20500f584eff1be1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:24.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.153+0000 7f26d5600640 1 -- 172.21.15.103:0/3045327122 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f26c4018a40 con 0x7f26d81047e0 2024-09-16T02:06:24.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.153+0000 7f26ca000640 1 -- 172.21.15.103:0/4043366577 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0040030 con 0x562e6db9fe40 2024-09-16T02:06:24.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.153+0000 7f26ca000640 1 -- 172.21.15.103:0/4043366577 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d0030ba0 con 0x562e6db9fe40 2024-09-16T02:06:24.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.154+0000 7f26ca000640 1 -- 172.21.15.103:0/4043366577 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0036040 con 0x562e6db9fe40 2024-09-16T02:06:24.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.154+0000 7f26e248c740 1 -- 172.21.15.103:0/4043366577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 msgr2=0x562e6dba0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:24.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.154+0000 7f26e248c740 1 --2- 172.21.15.103:0/4043366577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba0220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f26d002acb0 tx=0x7f26d0030a30 comp rx=0 tx=0).stop 2024-09-16T02:06:24.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.155+0000 7f26e248c740 1 -- 172.21.15.103:0/4043366577 shutdown_connections 2024-09-16T02:06:24.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.155+0000 7f26e248c740 1 --2- 172.21.15.103:0/4043366577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dbab020 0x562e6dbad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.155+0000 7f26e248c740 1 --2- 172.21.15.103:0/4043366577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dba0760 0x562e6dbaaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.155+0000 7f26e248c740 1 --2- 172.21.15.103:0/4043366577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba0220 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.155+0000 7f26e248c740 1 -- 172.21.15.103:0/4043366577 >> 172.21.15.103:0/4043366577 conn(0x562e6db9c3d0 msgr2=0x562e6db9c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:24.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.156+0000 7f26e248c740 1 -- 172.21.15.103:0/4043366577 shutdown_connections 2024-09-16T02:06:24.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.156+0000 7f26e248c740 1 -- 172.21.15.103:0/4043366577 wait complete. 2024-09-16T02:06:24.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.156+0000 7f26e248c740 1 -- start start 2024-09-16T02:06:24.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.157+0000 7f26e248c740 1 Processor -- start 2024-09-16T02:06:24.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.157+0000 7f26d5600640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f26c00a32e0 0x7f26c00a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.157+0000 7f26e248c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.157+0000 7f26e248c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 0x562e6dba83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.157+0000 7f26e248c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 0x562e6dba8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.157+0000 7f26e248c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562e6dbadc90 con 0x562e6dbab020 2024-09-16T02:06:24.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.157+0000 7f26e248c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562e6dac10f0 con 0x562e6dba0760 2024-09-16T02:06:24.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.158+0000 7f26e248c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562e6dbade10 con 0x562e6db9fe40 2024-09-16T02:06:24.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.158+0000 7f26e248c740 10 client.0 _ll_register_callbacks cb 0x562e6daa8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:24.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.158+0000 7f26d5600640 1 -- 172.21.15.103:0/3045327122 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f26c40cfe70 con 0x7f26d81047e0 2024-09-16T02:06:24.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.158+0000 7f26dda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 0x562e6dba8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26dd000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 0x562e6dba83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26dd000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:34864/0 (socket says 172.21.15.103:34864) 2024-09-16T02:06:24.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26dd000640 1 -- 172.21.15.103:0/213566743 learned_addr learned my addr 172.21.15.103:0/213566743 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:24.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26dda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 0x562e6dba8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47722/0 (socket says 172.21.15.103:47722) 2024-09-16T02:06:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26dda00640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f26c00a32e0 0x7f26c00a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26dd000640 1 -- 172.21.15.103:0/213566743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 msgr2=0x562e6dba83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26dd000640 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 0x562e6dba83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26dd000640 1 -- 172.21.15.103:0/213566743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 msgr2=0x562e6dba8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26dd000640 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 0x562e6dba8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.159+0000 7f26dd000640 1 -- 172.21.15.103:0/213566743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6d8773c0 con 0x562e6db9fe40 2024-09-16T02:06:24.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.160+0000 7f26dd000640 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba7ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26cc011720 tx=0x7f26cc011be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac50d49dac2ccbf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:24.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.160+0000 7f26bea00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:06:24.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.161+0000 7f26dda00640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f26c00a32e0 0x7f26c00a57a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f26d002abe0 tx=0x7f26d002a6c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:24.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.161+0000 7f26bea00640 1 -- 172.21.15.103:0/213566743 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc01f060 con 0x562e6db9fe40 2024-09-16T02:06:24.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.161+0000 7f26bea00640 1 -- 172.21.15.103:0/213566743 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f26cc004320 con 0x562e6db9fe40 2024-09-16T02:06:24.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.161+0000 7f26e248c740 1 -- 172.21.15.103:0/213566743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d0030ba0 con 0x562e6db9fe40 2024-09-16T02:06:24.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.162+0000 7f26bea00640 1 -- 172.21.15.103:0/213566743 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26cc00f9a0 con 0x562e6db9fe40 2024-09-16T02:06:24.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.162+0000 7f26e248c740 10 client.25195 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:24.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.162+0000 7f26e248c740 1 -- 172.21.15.103:0/213566743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562e6dbc2560 con 0x562e6db9fe40 2024-09-16T02:06:24.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.162+0000 7f26bea00640 1 -- 172.21.15.103:0/213566743 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d0030ba0 con 0x562e6db9fe40 2024-09-16T02:06:24.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.163+0000 7f26bea00640 1 -- 172.21.15.103:0/213566743 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f26cc024410 con 0x562e6db9fe40 2024-09-16T02:06:24.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.163+0000 7f26e248c740 10 client.25195 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:24.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.163+0000 7f26e248c740 10 client.25195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:24.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.163+0000 7f26e248c740 20 client.25195 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:24.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.163+0000 7f26e248c740 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x562e6dbc3a20 0x562e6dbc5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.163+0000 7f26e248c740 4 client.25195 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:06:24.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.163+0000 7f26e248c740 1 -- 172.21.15.103:0/213566743 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e6db13950 con 0x562e6dbc3a20 2024-09-16T02:06:24.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.164+0000 7f26d7e00640 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x562e6dbc3a20 0x562e6dbc5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.164+0000 7f26d7e00640 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x562e6dbc3a20 0x562e6dbc5e30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5e707b66dc048a server_cookie=225f7b77cbc87d99 in_seq=0 out_seq=0 2024-09-16T02:06:24.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.164+0000 7f26bea00640 10 client.25195 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:24.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.165+0000 7f26bea00640 1 -- 172.21.15.103:0/213566743 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e6db13950 con 0x562e6dbc3a20 2024-09-16T02:06:24.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.165+0000 7f26bea00640 10 client.25195 handle_command_reply: tid=0 2024-09-16T02:06:24.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.165+0000 7f26e248c740 1 client.25195 shutdown 2024-09-16T02:06:24.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 1 -- 172.21.15.103:0/213566743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 msgr2=0x562e6dba7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:24.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba7ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26cc011720 tx=0x7f26cc011be0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 1 -- 172.21.15.103:0/213566743 shutdown_connections 2024-09-16T02:06:24.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x562e6dbc3a20 0x562e6dbc5e30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 0x562e6dba8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 0x562e6dba83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 1 --2- 172.21.15.103:0/213566743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba7ea0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26bea00640 0 client.25195 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:24.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 1 -- 172.21.15.103:0/213566743 >> 172.21.15.103:0/213566743 conn(0x562e6db9c3d0 msgr2=0x562e6dba2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:24.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 1 -- 172.21.15.103:0/213566743 shutdown_connections 2024-09-16T02:06:24.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 1 -- 172.21.15.103:0/213566743 wait complete. 2024-09-16T02:06:24.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.166+0000 7f26e248c740 20 client.25195 trim_cache size 0 max 16384 2024-09-16T02:06:24.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.193+0000 7f26e248c740 1 Processor -- start 2024-09-16T02:06:24.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.193+0000 7f26e248c740 1 -- start start 2024-09-16T02:06:24.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.193+0000 7f26e248c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6db9fe40 0x562e6db9c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.193+0000 7f26e248c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6dba0760 0x562e6dbcb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.193+0000 7f26e248c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dbab020 0x562e6dbcbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.194+0000 7f26e248c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8010f40 con 0x562e6db9fe40 2024-09-16T02:06:24.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.194+0000 7f26e248c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f26b800e810 con 0x562e6dbab020 2024-09-16T02:06:24.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.194+0000 7f26e248c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26b800c660 con 0x562e6dba0760 2024-09-16T02:06:24.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.194+0000 7f26dd000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6db9fe40 0x562e6db9c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.194+0000 7f26dd000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6db9fe40 0x562e6db9c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47734/0 (socket says 172.21.15.103:47734) 2024-09-16T02:06:24.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.194+0000 7f26dd000640 1 -- 172.21.15.103:0/2126366659 learned_addr learned my addr 172.21.15.103:0/2126366659 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:24.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.194+0000 7f26d7e00640 1 --2- 172.21.15.103:0/2126366659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6dba0760 0x562e6dbcb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.194+0000 7f26dda00640 1 --2- 172.21.15.103:0/2126366659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dbab020 0x562e6dbcbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.194+0000 7f26d7e00640 1 -- 172.21.15.103:0/2126366659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dbab020 msgr2=0x562e6dbcbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:24.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.194+0000 7f26d7e00640 1 --2- 172.21.15.103:0/2126366659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dbab020 0x562e6dbcbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.195+0000 7f26d7e00640 1 -- 172.21.15.103:0/2126366659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6db9fe40 msgr2=0x562e6db9c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:24.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.195+0000 7f26d7e00640 1 --2- 172.21.15.103:0/2126366659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6db9fe40 0x562e6db9c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.195+0000 7f26d7e00640 1 -- 172.21.15.103:0/2126366659 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6dba6eb0 con 0x562e6dba0760 2024-09-16T02:06:24.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.195+0000 7f26d7e00640 1 --2- 172.21.15.103:0/2126366659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6dba0760 0x562e6dbcb740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f26c40cd740 tx=0x7f26c4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2ecb0c2d6e2297 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:24.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.195+0000 7f26c9600640 1 -- 172.21.15.103:0/2126366659 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40988c0 con 0x562e6dba0760 2024-09-16T02:06:24.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.195+0000 7f26c9600640 1 -- 172.21.15.103:0/2126366659 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c40cd990 con 0x562e6dba0760 2024-09-16T02:06:24.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.195+0000 7f26c9600640 1 -- 172.21.15.103:0/2126366659 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40bc310 con 0x562e6dba0760 2024-09-16T02:06:24.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.196+0000 7f26e248c740 1 -- 172.21.15.103:0/2126366659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6dba0760 msgr2=0x562e6dbcb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:24.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.196+0000 7f26e248c740 1 --2- 172.21.15.103:0/2126366659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6dba0760 0x562e6dbcb740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f26c40cd740 tx=0x7f26c4014000 comp rx=0 tx=0).stop 2024-09-16T02:06:24.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.196+0000 7f26e248c740 1 -- 172.21.15.103:0/2126366659 shutdown_connections 2024-09-16T02:06:24.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.196+0000 7f26e248c740 1 --2- 172.21.15.103:0/2126366659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dbab020 0x562e6dbcbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.196+0000 7f26e248c740 1 --2- 172.21.15.103:0/2126366659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6dba0760 0x562e6dbcb740 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.196+0000 7f26e248c740 1 --2- 172.21.15.103:0/2126366659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6db9fe40 0x562e6db9c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.196+0000 7f26e248c740 1 -- 172.21.15.103:0/2126366659 >> 172.21.15.103:0/2126366659 conn(0x562e6db9c3d0 msgr2=0x562e6dba20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:24.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.197+0000 7f26e248c740 1 -- 172.21.15.103:0/2126366659 shutdown_connections 2024-09-16T02:06:24.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.197+0000 7f26e248c740 1 -- 172.21.15.103:0/2126366659 wait complete. 2024-09-16T02:06:24.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.197+0000 7f26e248c740 1 -- start start 2024-09-16T02:06:24.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.198+0000 7f26e248c740 1 Processor -- start 2024-09-16T02:06:24.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.198+0000 7f26e248c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.198+0000 7f26e248c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 0x562e6dba5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.198+0000 7f26e248c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 0x562e6dba5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.198+0000 7f26e248c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26b800ccd0 con 0x562e6dbab020 2024-09-16T02:06:24.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.198+0000 7f26e248c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f26b800e140 con 0x562e6dba0760 2024-09-16T02:06:24.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.198+0000 7f26e248c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26b800d3a0 con 0x562e6db9fe40 2024-09-16T02:06:24.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.198+0000 7f26e248c740 10 client.0 _ll_register_callbacks cb 0x7f26cc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:24.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.199+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 0x562e6dba5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.199+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 0x562e6dba5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:50640/0 (socket says 172.21.15.103:50640) 2024-09-16T02:06:24.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.199+0000 7f26d7e00640 1 -- 172.21.15.103:0/3541770715 learned_addr learned my addr 172.21.15.103:0/3541770715 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:24.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.199+0000 7f26dd000640 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.199+0000 7f26dda00640 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 0x562e6dba5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.200+0000 7f26d7e00640 1 -- 172.21.15.103:0/3541770715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 msgr2=0x562e6dba5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:24.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.200+0000 7f26d7e00640 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.200+0000 7f26d7e00640 1 -- 172.21.15.103:0/3541770715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 msgr2=0x562e6dba5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:24.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.200+0000 7f26d7e00640 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 0x562e6dba5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.200+0000 7f26d7e00640 1 -- 172.21.15.103:0/3541770715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6dbb7110 con 0x562e6dba0760 2024-09-16T02:06:24.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.200+0000 7f26d7e00640 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 0x562e6dba5550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f26c40a4990 tx=0x7f26c409c150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78351c79ecee6cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:24.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.200+0000 7f26bea00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:06:24.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.201+0000 7f26bea00640 1 -- 172.21.15.103:0/3541770715 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40b4070 con 0x562e6dba0760 2024-09-16T02:06:24.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.201+0000 7f26bea00640 1 -- 172.21.15.103:0/3541770715 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f26c40bc0b0 con 0x562e6dba0760 2024-09-16T02:06:24.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.201+0000 7f26e248c740 1 -- 172.21.15.103:0/3541770715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c40cd990 con 0x562e6dba0760 2024-09-16T02:06:24.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.201+0000 7f26bea00640 1 -- 172.21.15.103:0/3541770715 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26c40cdb40 con 0x562e6dba0760 2024-09-16T02:06:24.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.201+0000 7f26e248c740 10 client.25220 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:24.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.201+0000 7f26e248c740 1 -- 172.21.15.103:0/3541770715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562e6dbb83d0 con 0x562e6dba0760 2024-09-16T02:06:24.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.201+0000 7f26bea00640 1 -- 172.21.15.103:0/3541770715 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c40cd990 con 0x562e6dba0760 2024-09-16T02:06:24.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.202+0000 7f26bea00640 1 -- 172.21.15.103:0/3541770715 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f26c409b6a0 con 0x562e6dba0760 2024-09-16T02:06:24.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.202+0000 7f26e248c740 10 client.25220 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:24.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.202+0000 7f26e248c740 10 client.25220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:24.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.202+0000 7f26e248c740 20 client.25220 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:24.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.202+0000 7f26e248c740 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x562e6dbb8a10 0x562e6dbbae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:24.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.202+0000 7f26e248c740 4 client.25220 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:06:24.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.202+0000 7f26e248c740 1 -- 172.21.15.103:0/3541770715 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e6db0de60 con 0x562e6dbb8a10 2024-09-16T02:06:24.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.203+0000 7f26dd000640 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x562e6dbb8a10 0x562e6dbbae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:24.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.204+0000 7f26dd000640 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x562e6dbb8a10 0x562e6dbbae30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb48099c7fcbf29 server_cookie=17010eab30d05953 in_seq=0 out_seq=0 2024-09-16T02:06:24.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.204+0000 7f26bea00640 10 client.25220 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:24.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.204+0000 7f26bea00640 1 -- 172.21.15.103:0/3541770715 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e6db0de60 con 0x562e6dbb8a10 2024-09-16T02:06:24.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.204+0000 7f26bea00640 10 client.25220 handle_command_reply: tid=0 2024-09-16T02:06:24.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.204+0000 7f26e248c740 1 client.25220 shutdown 2024-09-16T02:06:24.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.205+0000 7f26e248c740 1 -- 172.21.15.103:0/3541770715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 msgr2=0x562e6dba5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:24.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.205+0000 7f26e248c740 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 0x562e6dba5550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f26c40a4990 tx=0x7f26c409c150 comp rx=0 tx=0).stop 2024-09-16T02:06:24.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.205+0000 7f26e248c740 1 -- 172.21.15.103:0/3541770715 shutdown_connections 2024-09-16T02:06:24.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.205+0000 7f26e248c740 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x562e6dbb8a10 0x562e6dbbae30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.205+0000 7f26e248c740 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6dbab020 0x562e6dba5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.205+0000 7f26e248c740 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562e6dba0760 0x562e6dba5550 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.205+0000 7f26e248c740 1 --2- 172.21.15.103:0/3541770715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562e6db9fe40 0x562e6dba5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.205+0000 7f26e248c740 1 -- 172.21.15.103:0/3541770715 >> 172.21.15.103:0/3541770715 conn(0x562e6db9c3d0 msgr2=0x562e6dbbe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:24.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.205+0000 7f26bea00640 0 client.25220 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:24.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.205+0000 7f26e248c740 1 -- 172.21.15.103:0/3541770715 shutdown_connections 2024-09-16T02:06:24.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.206+0000 7f26e248c740 1 -- 172.21.15.103:0/3541770715 wait complete. 2024-09-16T02:06:24.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.206+0000 7f26e248c740 20 client.25220 trim_cache size 0 max 16384 2024-09-16T02:06:24.209 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-16T02:06:24.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 -- 172.21.15.103:0/3045327122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f26c00a32e0 msgr2=0x7f26c00a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:24.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f26c00a32e0 0x7f26c00a57a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f26d002abe0 tx=0x7f26d002a6c0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 -- 172.21.15.103:0/3045327122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d81047e0 msgr2=0x7f26d8073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:24.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d81047e0 0x7f26d8073ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26c400e9f0 tx=0x7f26c400eeb0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 -- 172.21.15.103:0/3045327122 shutdown_connections 2024-09-16T02:06:24.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f26c00a32e0 0x7f26c00a57a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26d8108f60 0x7f26d806cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d81047e0 0x7f26d8073ee0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 --2- 172.21.15.103:0/3045327122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8103e30 0x7f26d80739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:24.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 -- 172.21.15.103:0/3045327122 >> 172.21.15.103:0/3045327122 conn(0x7f26d80ff910 msgr2=0x7f26d8100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:24.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 -- 172.21.15.103:0/3045327122 shutdown_connections 2024-09-16T02:06:24.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:24.207+0000 7f26de400640 1 -- 172.21.15.103:0/3045327122 wait complete. 2024-09-16T02:06:26.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:25 smithi103 ceph-mon[30398]: pgmap v369: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T02:06:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:25 smithi136 ceph-mon[35323]: pgmap v369: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T02:06:26.031 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T02:06:26.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T02:06:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:25 smithi172 ceph-mon[35340]: pgmap v369: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T02:06:26.489 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:27.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.087+0000 7f2aa1a00640 1 -- 172.21.15.103:0/36316103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a9c1034b0 msgr2=0x7f2a9c10b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.087+0000 7f2aa1a00640 1 --2- 172.21.15.103:0/36316103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a9c1034b0 0x7f2a9c10b9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009860 tx=0x7f2a8802a3d0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.087+0000 7f2aa1a00640 1 -- 172.21.15.103:0/36316103 shutdown_connections 2024-09-16T02:06:27.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.087+0000 7f2aa1a00640 1 --2- 172.21.15.103:0/36316103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c10bfc0 0x7f2a9c10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.087+0000 7f2aa1a00640 1 --2- 172.21.15.103:0/36316103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a9c1034b0 0x7f2a9c10b9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.087+0000 7f2aa1a00640 1 --2- 172.21.15.103:0/36316103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c102b90 0x7f2a9c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.087+0000 7f2aa1a00640 1 -- 172.21.15.103:0/36316103 >> 172.21.15.103:0/36316103 conn(0x7f2a9c0fc610 msgr2=0x7f2a9c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:27.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.087+0000 7f2aa1a00640 1 -- 172.21.15.103:0/36316103 shutdown_connections 2024-09-16T02:06:27.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.087+0000 7f2aa1a00640 1 -- 172.21.15.103:0/36316103 wait complete. 2024-09-16T02:06:27.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.088+0000 7f2aa1a00640 1 Processor -- start 2024-09-16T02:06:27.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.088+0000 7f2aa1a00640 1 -- start start 2024-09-16T02:06:27.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.089+0000 7f2aa1a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a9c102b90 0x7f2a9c0739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.089+0000 7f2aa1a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c1034b0 0x7f2a9c06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.089+0000 7f2aa1a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c10bfc0 0x7f2a9c06cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.089+0000 7f2aa1a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c10fc20 con 0x7f2a9c10bfc0 2024-09-16T02:06:27.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.089+0000 7f2aa1a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c10faa0 con 0x7f2a9c1034b0 2024-09-16T02:06:27.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.089+0000 7f2aa1a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c10fda0 con 0x7f2a9c102b90 2024-09-16T02:06:27.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.089+0000 7f2a9be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a9c102b90 0x7f2a9c0739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.089+0000 7f2a9be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a9c102b90 0x7f2a9c0739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:34916/0 (socket says 172.21.15.103:34916) 2024-09-16T02:06:27.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.089+0000 7f2a9be00640 1 -- 172.21.15.103:0/4048680618 learned_addr learned my addr 172.21.15.103:0/4048680618 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:27.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.089+0000 7f2a9b400640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c1034b0 0x7f2a9c06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.090+0000 7f2aa1000640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c10bfc0 0x7f2a9c06cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.090+0000 7f2a9be00640 1 -- 172.21.15.103:0/4048680618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c1034b0 msgr2=0x7f2a9c06ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.090+0000 7f2a9be00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c1034b0 0x7f2a9c06ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.090+0000 7f2a9be00640 1 -- 172.21.15.103:0/4048680618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c10bfc0 msgr2=0x7f2a9c06cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.090+0000 7f2a9be00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c10bfc0 0x7f2a9c06cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.090+0000 7f2a9be00640 1 -- 172.21.15.103:0/4048680618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c06d500 con 0x7f2a9c102b90 2024-09-16T02:06:27.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.091+0000 7f2a9be00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a9c102b90 0x7f2a9c0739b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00bf10 tx=0x7f2a8c0095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:27.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.091+0000 7f2a98c00640 1 -- 172.21.15.103:0/4048680618 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c01a020 con 0x7f2a9c102b90 2024-09-16T02:06:27.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.091+0000 7f2a98c00640 1 -- 172.21.15.103:0/4048680618 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c00d460 con 0x7f2a9c102b90 2024-09-16T02:06:27.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.091+0000 7f2aa1a00640 1 -- 172.21.15.103:0/4048680618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9c06d7f0 con 0x7f2a9c102b90 2024-09-16T02:06:27.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.091+0000 7f2a98c00640 1 -- 172.21.15.103:0/4048680618 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c0157c0 con 0x7f2a9c102b90 2024-09-16T02:06:27.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.091+0000 7f2aa1a00640 1 -- 172.21.15.103:0/4048680618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9c0717c0 con 0x7f2a9c102b90 2024-09-16T02:06:27.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.095+0000 7f2aa59be740 1 Processor -- start 2024-09-16T02:06:27.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.095+0000 7f2aa59be740 1 -- start start 2024-09-16T02:06:27.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.095+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699efcb0 0x5588699f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.095+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699f05d0 0x5588699fa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.095+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x5588699fd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.095+0000 7f2aa59be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558869956290 con 0x5588699f05d0 2024-09-16T02:06:27.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.095+0000 7f2aa59be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5588698fdb40 con 0x5588699efcb0 2024-09-16T02:06:27.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.095+0000 7f2aa59be740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558869952250 con 0x5588699fae90 2024-09-16T02:06:27.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.096+0000 7f2aa1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699efcb0 0x5588699f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.096+0000 7f2aa1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699efcb0 0x5588699f0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:50664/0 (socket says 172.21.15.103:50664) 2024-09-16T02:06:27.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.096+0000 7f2aa1000640 1 -- 172.21.15.103:0/2513177492 learned_addr learned my addr 172.21.15.103:0/2513177492 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:27.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.096+0000 7f2a9b400640 1 --2- 172.21.15.103:0/2513177492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x5588699fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.096+0000 7f2aa1000640 1 -- 172.21.15.103:0/2513177492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 msgr2=0x5588699fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:27.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.096+0000 7f2aa1000640 1 --2- 172.21.15.103:0/2513177492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x5588699fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.096+0000 7f2aa1000640 1 -- 172.21.15.103:0/2513177492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699f05d0 msgr2=0x5588699fa950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T02:06:27.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.096+0000 7f2aa1000640 1 --2- 172.21.15.103:0/2513177492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699f05d0 0x5588699fa950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.096+0000 7f2aa1000640 1 -- 172.21.15.103:0/2513177492 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588696c76a0 con 0x5588699efcb0 2024-09-16T02:06:27.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.097+0000 7f2a98c00640 1 -- 172.21.15.103:0/4048680618 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f2a8c015960 con 0x7f2a9c102b90 2024-09-16T02:06:27.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.097+0000 7f2aa1000640 1 --2- 172.21.15.103:0/2513177492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699efcb0 0x5588699f0090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400b6b0 tx=0x7f2a94004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e44b51ee88da5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:27.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.097+0000 7f2a91600640 1 -- 172.21.15.103:0/2513177492 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94003b20 con 0x5588699efcb0 2024-09-16T02:06:27.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.097+0000 7f2a91600640 1 -- 172.21.15.103:0/2513177492 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a94015070 con 0x5588699efcb0 2024-09-16T02:06:27.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.097+0000 7f2a91600640 1 -- 172.21.15.103:0/2513177492 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940116a0 con 0x5588699efcb0 2024-09-16T02:06:27.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.098+0000 7f2a98c00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2a800a3350 0x7f2a800a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.098+0000 7f2aa59be740 1 -- 172.21.15.103:0/2513177492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699efcb0 msgr2=0x5588699f0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.098+0000 7f2aa59be740 1 --2- 172.21.15.103:0/2513177492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699efcb0 0x5588699f0090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400b6b0 tx=0x7f2a94004610 comp rx=0 tx=0).stop 2024-09-16T02:06:27.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.098+0000 7f2a98c00640 1 -- 172.21.15.103:0/4048680618 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f2a8c0cfd20 con 0x7f2a9c102b90 2024-09-16T02:06:27.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.099+0000 7f2a9be00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2a800a3350 0x7f2a800a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.099+0000 7f2aa59be740 1 -- 172.21.15.103:0/2513177492 shutdown_connections 2024-09-16T02:06:27.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.099+0000 7f2aa59be740 1 --2- 172.21.15.103:0/2513177492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x5588699fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.099+0000 7f2aa59be740 1 --2- 172.21.15.103:0/2513177492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699f05d0 0x5588699fa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.099+0000 7f2aa59be740 1 --2- 172.21.15.103:0/2513177492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699efcb0 0x5588699f0090 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.099+0000 7f2aa59be740 1 -- 172.21.15.103:0/2513177492 >> 172.21.15.103:0/2513177492 conn(0x5588699ec240 msgr2=0x5588699ec620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:27.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.099+0000 7f2aa59be740 1 -- 172.21.15.103:0/2513177492 shutdown_connections 2024-09-16T02:06:27.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.099+0000 7f2aa59be740 1 -- 172.21.15.103:0/2513177492 wait complete. 2024-09-16T02:06:27.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.100+0000 7f2aa59be740 1 -- start start 2024-09-16T02:06:27.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.100+0000 7f2aa59be740 1 Processor -- start 2024-09-16T02:06:27.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.100+0000 7f2a9be00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2a800a3350 0x7f2a800a5810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c098b60 tx=0x7f2a8c009f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:27.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.101+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.101+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699f05d0 0x5588699f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.101+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699fae90 0x5588699f8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.101+0000 7f2aa59be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5588699fdb00 con 0x5588699efcb0 2024-09-16T02:06:27.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.101+0000 7f2aa59be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558869910fa0 con 0x5588699fae90 2024-09-16T02:06:27.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.101+0000 7f2aa59be740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5588699fdc80 con 0x5588699f05d0 2024-09-16T02:06:27.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.101+0000 7f2aa59be740 10 client.0 _ll_register_callbacks cb 0x558869664ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:27.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.101+0000 7f2aa1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.101+0000 7f2aa1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699fa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47766/0 (socket says 172.21.15.103:47766) 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.101+0000 7f2aa1000640 1 -- 172.21.15.103:0/2190245036 learned_addr learned my addr 172.21.15.103:0/2190245036 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.102+0000 7f2aa1000640 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699fae90 0x5588699f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.102+0000 7f2a9b400640 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699f05d0 0x5588699f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.102+0000 7f2aa1000640 1 -- 172.21.15.103:0/2190245036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699f05d0 msgr2=0x5588699f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.102+0000 7f2aa1000640 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699f05d0 0x5588699f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.102+0000 7f2aa1000640 1 -- 172.21.15.103:0/2190245036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699fae90 msgr2=0x5588699f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.102+0000 7f2aa1000640 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699fae90 0x5588699f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.102+0000 7f2aa1000640 1 -- 172.21.15.103:0/2190245036 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588699f8830 con 0x5588699efcb0 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.102+0000 7f2aa1000640 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699fa7f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a940057c0 tx=0x7f2a940056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd7cbb4839f0162 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.102+0000 7f2a86000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:06:27.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.102+0000 7f2a86000640 1 -- 172.21.15.103:0/2190245036 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94003b20 con 0x5588699efcb0 2024-09-16T02:06:27.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.103+0000 7f2a86000640 1 -- 172.21.15.103:0/2190245036 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f2a94003cc0 con 0x5588699efcb0 2024-09-16T02:06:27.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.103+0000 7f2aa59be740 1 -- 172.21.15.103:0/2190245036 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a94015070 con 0x5588699efcb0 2024-09-16T02:06:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.103+0000 7f2a86000640 1 -- 172.21.15.103:0/2190245036 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2a94003ec0 con 0x5588699efcb0 2024-09-16T02:06:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.103+0000 7f2aa59be740 10 client.15282 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.103+0000 7f2aa59be740 1 -- 172.21.15.103:0/2190245036 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558869a09a40 con 0x5588699efcb0 2024-09-16T02:06:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.103+0000 7f2a86000640 1 -- 172.21.15.103:0/2190245036 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a94015070 con 0x5588699efcb0 2024-09-16T02:06:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.104+0000 7f2a86000640 1 -- 172.21.15.103:0/2190245036 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2a94027020 con 0x5588699efcb0 2024-09-16T02:06:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.104+0000 7f2aa59be740 10 client.15282 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.104+0000 7f2aa59be740 10 client.15282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.104+0000 7f2aa59be740 20 client.15282 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.104+0000 7f2aa59be740 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x558869a0ae50 0x558869a0d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.104+0000 7f2aa59be740 4 client.15282 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:06:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.104+0000 7f2aa59be740 1 -- 172.21.15.103:0/2190245036 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558869963800 con 0x558869a0ae50 2024-09-16T02:06:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.104+0000 7f2a9b400640 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x558869a0ae50 0x558869a0d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.105+0000 7f2a86000640 10 client.15282 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.105+0000 7f2a9b400640 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x558869a0ae50 0x558869a0d240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3ebd0602e01ace server_cookie=c416cd5a59afc3a4 in_seq=0 out_seq=0 2024-09-16T02:06:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2a86000640 1 -- 172.21.15.103:0/2190245036 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558869963800 con 0x558869a0ae50 2024-09-16T02:06:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2a86000640 10 client.15282 handle_command_reply: tid=0 2024-09-16T02:06:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2aa59be740 1 client.15282 shutdown 2024-09-16T02:06:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2aa59be740 1 -- 172.21.15.103:0/2190245036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 msgr2=0x5588699fa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2aa59be740 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699fa7f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a940057c0 tx=0x7f2a940056d0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2aa59be740 1 -- 172.21.15.103:0/2190245036 shutdown_connections 2024-09-16T02:06:27.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2aa59be740 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x558869a0ae50 0x558869a0d240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2aa59be740 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699fae90 0x5588699f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2a86000640 0 client.15282 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2aa59be740 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699f05d0 0x5588699f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2aa59be740 1 --2- 172.21.15.103:0/2190245036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699fa7f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.106+0000 7f2aa59be740 1 -- 172.21.15.103:0/2190245036 >> 172.21.15.103:0/2190245036 conn(0x5588699ec240 msgr2=0x5588699f1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.107+0000 7f2aa59be740 1 -- 172.21.15.103:0/2190245036 shutdown_connections 2024-09-16T02:06:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.107+0000 7f2aa59be740 1 -- 172.21.15.103:0/2190245036 wait complete. 2024-09-16T02:06:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.108+0000 7f2aa59be740 20 client.15282 trim_cache size 0 max 16384 2024-09-16T02:06:27.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.126+0000 7f2aa59be740 1 Processor -- start 2024-09-16T02:06:27.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.126+0000 7f2aa59be740 1 -- start start 2024-09-16T02:06:27.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.126+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.126+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 0x558869a10270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.126+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x558869a107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.126+0000 7f2aa59be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1c76b0 con 0x5588699efcb0 2024-09-16T02:06:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.126+0000 7f2aa59be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1adf30 con 0x5588699f05d0 2024-09-16T02:06:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.126+0000 7f2aa59be740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1abd80 con 0x5588699fae90 2024-09-16T02:06:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.127+0000 7f2aa1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.127+0000 7f2aa1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47774/0 (socket says 172.21.15.103:47774) 2024-09-16T02:06:27.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.127+0000 7f2aa1000640 1 -- 172.21.15.103:0/3343200247 learned_addr learned my addr 172.21.15.103:0/3343200247 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:27.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.127+0000 7f2a9b400640 1 --2- 172.21.15.103:0/3343200247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 0x558869a10270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.127+0000 7f2aa1000640 1 --2- 172.21.15.103:0/3343200247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x558869a107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.128+0000 7f2aa1000640 1 -- 172.21.15.103:0/3343200247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 msgr2=0x558869a107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:27.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.128+0000 7f2aa1000640 1 --2- 172.21.15.103:0/3343200247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x558869a107b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.128+0000 7f2aa1000640 1 -- 172.21.15.103:0/3343200247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 msgr2=0x558869a10270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:27.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.128+0000 7f2aa1000640 1 --2- 172.21.15.103:0/3343200247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 0x558869a10270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.128+0000 7f2aa1000640 1 -- 172.21.15.103:0/3343200247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588699ed2f0 con 0x5588699efcb0 2024-09-16T02:06:27.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.128+0000 7f2a9b400640 1 --2- 172.21.15.103:0/3343200247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 0x558869a10270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:27.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.128+0000 7f2aa1000640 1 --2- 172.21.15.103:0/3343200247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699f7a10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2a94011900 tx=0x7f2a9401aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feedb44d54002f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:27.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.129+0000 7f2a90c00640 1 -- 172.21.15.103:0/3343200247 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9401a7e0 con 0x5588699efcb0 2024-09-16T02:06:27.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.129+0000 7f2a90c00640 1 -- 172.21.15.103:0/3343200247 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a94015070 con 0x5588699efcb0 2024-09-16T02:06:27.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.129+0000 7f2a90c00640 1 -- 172.21.15.103:0/3343200247 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94005040 con 0x5588699efcb0 2024-09-16T02:06:27.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.129+0000 7f2aa59be740 1 -- 172.21.15.103:0/3343200247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 msgr2=0x5588699f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.129+0000 7f2aa59be740 1 --2- 172.21.15.103:0/3343200247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699f7a10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2a94011900 tx=0x7f2a9401aeb0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.130+0000 7f2aa59be740 1 -- 172.21.15.103:0/3343200247 shutdown_connections 2024-09-16T02:06:27.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.130+0000 7f2aa59be740 1 --2- 172.21.15.103:0/3343200247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x558869a107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.130+0000 7f2aa59be740 1 --2- 172.21.15.103:0/3343200247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 0x558869a10270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.130+0000 7f2aa59be740 1 --2- 172.21.15.103:0/3343200247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x5588699f7a10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.130+0000 7f2aa59be740 1 -- 172.21.15.103:0/3343200247 >> 172.21.15.103:0/3343200247 conn(0x5588699ec240 msgr2=0x5588699f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:27.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.130+0000 7f2aa59be740 1 -- 172.21.15.103:0/3343200247 shutdown_connections 2024-09-16T02:06:27.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.130+0000 7f2aa59be740 1 -- 172.21.15.103:0/3343200247 wait complete. 2024-09-16T02:06:27.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.130+0000 7f2aa59be740 1 -- start start 2024-09-16T02:06:27.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.131+0000 7f2aa59be740 1 Processor -- start 2024-09-16T02:06:27.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.131+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x558869a10060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.131+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 0x558869a105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.131+0000 7f2aa59be740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x558869a6cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.131+0000 7f2aa59be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1ac3f0 con 0x5588699efcb0 2024-09-16T02:06:27.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.131+0000 7f2aa59be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1ad860 con 0x5588699f05d0 2024-09-16T02:06:27.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.131+0000 7f2aa59be740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1acac0 con 0x5588699fae90 2024-09-16T02:06:27.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.131+0000 7f2aa59be740 10 client.0 _ll_register_callbacks cb 0x5588699f8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:27.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.131+0000 7f2aa1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x558869a10060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.132+0000 7f2a9b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 0x558869a105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.132+0000 7f2aa1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x558869a10060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47784/0 (socket says 172.21.15.103:47784) 2024-09-16T02:06:27.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.132+0000 7f2aa1000640 1 -- 172.21.15.103:0/3762621338 learned_addr learned my addr 172.21.15.103:0/3762621338 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:27.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.132+0000 7f2aa1000640 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x558869a6cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.132+0000 7f2a9b400640 1 -- 172.21.15.103:0/3762621338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 msgr2=0x558869a6cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:27.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.132+0000 7f2a9b400640 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x558869a6cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.132+0000 7f2a9b400640 1 -- 172.21.15.103:0/3762621338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 msgr2=0x558869a10060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:27.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.132+0000 7f2a9b400640 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x558869a10060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.132+0000 7f2a9b400640 1 -- 172.21.15.103:0/3762621338 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558869a6d320 con 0x5588699f05d0 2024-09-16T02:06:27.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.132+0000 7f2aa1000640 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x558869a10060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:27.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.133+0000 7f2a9b400640 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 0x558869a105a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009830 tx=0x7f2a88030ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f259637e4bc448cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:27.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.133+0000 7f2a86000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:06:27.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.134+0000 7f2a86000640 1 -- 172.21.15.103:0/3762621338 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a880303a0 con 0x5588699f05d0 2024-09-16T02:06:27.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.134+0000 7f2a86000640 1 -- 172.21.15.103:0/3762621338 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f2a88030540 con 0x5588699f05d0 2024-09-16T02:06:27.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.134+0000 7f2aa59be740 1 -- 172.21.15.103:0/3762621338 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a94015070 con 0x5588699f05d0 2024-09-16T02:06:27.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.134+0000 7f2a86000640 1 -- 172.21.15.103:0/3762621338 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2a88056020 con 0x5588699f05d0 2024-09-16T02:06:27.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.134+0000 7f2aa59be740 10 client.25238 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:27.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.134+0000 7f2aa59be740 1 -- 172.21.15.103:0/3762621338 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558869a75930 con 0x5588699f05d0 2024-09-16T02:06:27.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.134+0000 7f2a86000640 1 -- 172.21.15.103:0/3762621338 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a94015070 con 0x5588699f05d0 2024-09-16T02:06:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.135+0000 7f2a86000640 1 -- 172.21.15.103:0/3762621338 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2a88048410 con 0x5588699f05d0 2024-09-16T02:06:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.135+0000 7f2aa59be740 10 client.25238 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.135+0000 7f2aa59be740 10 client.25238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.135+0000 7f2aa59be740 20 client.25238 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.135+0000 7f2aa59be740 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x558869a76090 0x558869a784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.135+0000 7f2aa59be740 4 client.25238 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T02:06:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.135+0000 7f2aa59be740 1 -- 172.21.15.103:0/3762621338 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55886995dd10 con 0x558869a76090 2024-09-16T02:06:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.135+0000 7f2aa1000640 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x558869a76090 0x558869a784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.136+0000 7f2aa1000640 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x558869a76090 0x558869a784b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7297ae48bf8751 server_cookie=540a7a31278fc1f7 in_seq=0 out_seq=0 2024-09-16T02:06:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.136+0000 7f2a86000640 10 client.25238 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:27.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.137+0000 7f2a86000640 1 -- 172.21.15.103:0/3762621338 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55886995dd10 con 0x558869a76090 2024-09-16T02:06:27.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.137+0000 7f2a86000640 10 client.25238 handle_command_reply: tid=0 2024-09-16T02:06:27.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.137+0000 7f2aa59be740 1 client.25238 shutdown 2024-09-16T02:06:27.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 1 -- 172.21.15.103:0/3762621338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 msgr2=0x558869a105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 0x558869a105a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009830 tx=0x7f2a88030ee0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 1 -- 172.21.15.103:0/3762621338 shutdown_connections 2024-09-16T02:06:27.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x558869a76090 0x558869a784b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588699fae90 0x558869a6cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588699f05d0 0x558869a105a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 1 --2- 172.21.15.103:0/3762621338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588699efcb0 0x558869a10060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 1 -- 172.21.15.103:0/3762621338 >> 172.21.15.103:0/3762621338 conn(0x5588699ec240 msgr2=0x5588699eaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:27.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2a86000640 0 client.25238 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:27.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 1 -- 172.21.15.103:0/3762621338 shutdown_connections 2024-09-16T02:06:27.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 1 -- 172.21.15.103:0/3762621338 wait complete. 2024-09-16T02:06:27.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.138+0000 7f2aa59be740 20 client.25238 trim_cache size 0 max 16384 2024-09-16T02:06:27.142 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:06:27.142 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-16T02:06:27.142 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "da49be25-2a4c-4d26-b93b-9a5eca4137d0", 2024-09-16T02:06:27.142 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-16T02:06:27.142 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:06:27.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.139+0000 7f2aa1a00640 1 -- 172.21.15.103:0/4048680618 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2a800a3350 msgr2=0x7f2a800a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.139+0000 7f2aa1a00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2a800a3350 0x7f2a800a5810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c098b60 tx=0x7f2a8c009f30 comp rx=0 tx=0).stop 2024-09-16T02:06:27.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.139+0000 7f2aa1a00640 1 -- 172.21.15.103:0/4048680618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a9c102b90 msgr2=0x7f2a9c0739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:27.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.139+0000 7f2aa1a00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a9c102b90 0x7f2a9c0739b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00bf10 tx=0x7f2a8c0095b0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.140+0000 7f2aa1a00640 1 -- 172.21.15.103:0/4048680618 shutdown_connections 2024-09-16T02:06:27.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.140+0000 7f2aa1a00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f2a800a3350 0x7f2a800a5810 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.140+0000 7f2aa1a00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c10bfc0 0x7f2a9c06cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.140+0000 7f2aa1a00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c1034b0 0x7f2a9c06ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.140+0000 7f2aa1a00640 1 --2- 172.21.15.103:0/4048680618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a9c102b90 0x7f2a9c0739b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:27.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.140+0000 7f2aa1a00640 1 -- 172.21.15.103:0/4048680618 >> 172.21.15.103:0/4048680618 conn(0x7f2a9c0fc610 msgr2=0x7f2a9c0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:27.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.140+0000 7f2aa1a00640 1 -- 172.21.15.103:0/4048680618 shutdown_connections 2024-09-16T02:06:27.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:27.140+0000 7f2aa1a00640 1 -- 172.21.15.103:0/4048680618 wait complete. 2024-09-16T02:06:28.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:27 smithi103 ceph-mon[30398]: pgmap v370: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-16T02:06:28.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:27 smithi103 ceph-mon[30398]: scrub queued for path: / 2024-09-16T02:06:28.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:27 smithi103 ceph-mon[30398]: scrub summary: idle+waiting paths [/] 2024-09-16T02:06:28.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:27 smithi103 ceph-mon[30398]: scrub summary: active paths [/] 2024-09-16T02:06:28.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:27 smithi103 ceph-mon[30398]: scrub summary: idle+waiting paths [/] 2024-09-16T02:06:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:27 smithi136 ceph-mon[35323]: pgmap v370: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-16T02:06:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:27 smithi136 ceph-mon[35323]: scrub queued for path: / 2024-09-16T02:06:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:27 smithi136 ceph-mon[35323]: scrub summary: idle+waiting paths [/] 2024-09-16T02:06:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:27 smithi136 ceph-mon[35323]: scrub summary: active paths [/] 2024-09-16T02:06:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:27 smithi136 ceph-mon[35323]: scrub summary: idle+waiting paths [/] 2024-09-16T02:06:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:27 smithi172 ceph-mon[35340]: pgmap v370: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-16T02:06:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:27 smithi172 ceph-mon[35340]: scrub queued for path: / 2024-09-16T02:06:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:27 smithi172 ceph-mon[35340]: scrub summary: idle+waiting paths [/] 2024-09-16T02:06:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:27 smithi172 ceph-mon[35340]: scrub summary: active paths [/] 2024-09-16T02:06:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:27 smithi172 ceph-mon[35340]: scrub summary: idle+waiting paths [/] 2024-09-16T02:06:28.121 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:06:28.559 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:06:29.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.122+0000 7f66c2c00640 1 -- 172.21.15.103:0/3030229447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66bc10e2d0 msgr2=0x7f66bc1106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.122+0000 7f66c2c00640 1 --2- 172.21.15.103:0/3030229447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66bc10e2d0 0x7f66bc1106c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009ca0 tx=0x7f66b402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.122+0000 7f66c2c00640 1 -- 172.21.15.103:0/3030229447 shutdown_connections 2024-09-16T02:06:29.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.122+0000 7f66c2c00640 1 --2- 172.21.15.103:0/3030229447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66bc10e2d0 0x7f66bc1106c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.122+0000 7f66c2c00640 1 --2- 172.21.15.103:0/3030229447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66bc1010d0 0x7f66bc10dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.122+0000 7f66c2c00640 1 --2- 172.21.15.103:0/3030229447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66bc100720 0x7f66bc100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.122+0000 7f66c2c00640 1 -- 172.21.15.103:0/3030229447 >> 172.21.15.103:0/3030229447 conn(0x7f66bc0fc610 msgr2=0x7f66bc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:29.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.122+0000 7f66c2c00640 1 -- 172.21.15.103:0/3030229447 shutdown_connections 2024-09-16T02:06:29.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.122+0000 7f66c2c00640 1 -- 172.21.15.103:0/3030229447 wait complete. 2024-09-16T02:06:29.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.124+0000 7f66c2c00640 1 Processor -- start 2024-09-16T02:06:29.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.124+0000 7f66c2c00640 1 -- start start 2024-09-16T02:06:29.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.124+0000 7f66c2c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66bc100720 0x7f66bc107390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.124+0000 7f66c2c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66bc1010d0 0x7f66bc1078d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.124+0000 7f66c2c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66bc10e2d0 0x7f66bc102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.124+0000 7f66c2c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc114020 con 0x7f66bc10e2d0 2024-09-16T02:06:29.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.124+0000 7f66c2c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc113ea0 con 0x7f66bc100720 2024-09-16T02:06:29.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.124+0000 7f66c2c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc1141a0 con 0x7f66bc1010d0 2024-09-16T02:06:29.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.125+0000 7f66c1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66bc100720 0x7f66bc107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.125+0000 7f66c1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66bc100720 0x7f66bc107390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:48460/0 (socket says 172.21.15.103:48460) 2024-09-16T02:06:29.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.125+0000 7f66c1800640 1 -- 172.21.15.103:0/673566455 learned_addr learned my addr 172.21.15.103:0/673566455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:29.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.125+0000 7f66c0e00640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66bc1010d0 0x7f66bc1078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.125+0000 7f66c2200640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66bc10e2d0 0x7f66bc102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.126+0000 7f66c1800640 1 -- 172.21.15.103:0/673566455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66bc1010d0 msgr2=0x7f66bc1078d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.126+0000 7f66c1800640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66bc1010d0 0x7f66bc1078d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.126+0000 7f66c1800640 1 -- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66bc10e2d0 msgr2=0x7f66bc102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.126+0000 7f66c1800640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66bc10e2d0 0x7f66bc102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.126+0000 7f66c1800640 1 -- 172.21.15.103:0/673566455 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc1031d0 con 0x7f66bc100720 2024-09-16T02:06:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.126+0000 7f66c2200640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66bc10e2d0 0x7f66bc102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:06:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.126+0000 7f66c1800640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66bc100720 0x7f66bc107390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f66b000e9f0 tx=0x7f66b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.127+0000 7f66ba000640 1 -- 172.21.15.103:0/673566455 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000cd80 con 0x7f66bc100720 2024-09-16T02:06:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.127+0000 7f66ba000640 1 -- 172.21.15.103:0/673566455 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66b0010910 con 0x7f66bc100720 2024-09-16T02:06:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.127+0000 7f66ba000640 1 -- 172.21.15.103:0/673566455 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0018990 con 0x7f66bc100720 2024-09-16T02:06:29.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.127+0000 7f66c2c00640 1 -- 172.21.15.103:0/673566455 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc1034c0 con 0x7f66bc100720 2024-09-16T02:06:29.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.127+0000 7f66c2c00640 1 -- 172.21.15.103:0/673566455 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc10d2f0 con 0x7f66bc100720 2024-09-16T02:06:29.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.130+0000 7f66c6e25740 1 Processor -- start 2024-09-16T02:06:29.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.130+0000 7f66c6e25740 1 -- start start 2024-09-16T02:06:29.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.131+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563834dd50 0x55563834e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.131+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834e670 0x5556383589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.131+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555638358f30 0x55563835b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.131+0000 7f66c6e25740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5556382b43e0 con 0x55563834e670 2024-09-16T02:06:29.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.131+0000 7f66c6e25740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55563825bd80 con 0x55563834dd50 2024-09-16T02:06:29.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.131+0000 7f66c6e25740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5556382b03a0 con 0x555638358f30 2024-09-16T02:06:29.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.131+0000 7f66c2200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563834dd50 0x55563834e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.131+0000 7f66c0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555638358f30 0x55563835b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.131+0000 7f66c2200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563834dd50 0x55563834e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:48476/0 (socket says 172.21.15.103:48476) 2024-09-16T02:06:29.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.131+0000 7f66c2200640 1 -- 172.21.15.103:0/2719295507 learned_addr learned my addr 172.21.15.103:0/2719295507 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:29.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.132+0000 7f66c0e00640 1 -- 172.21.15.103:0/2719295507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563834dd50 msgr2=0x55563834e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:29.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.132+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2719295507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563834dd50 0x55563834e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.132+0000 7f66c0e00640 1 -- 172.21.15.103:0/2719295507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834e670 msgr2=0x5556383589f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T02:06:29.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.132+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2719295507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834e670 0x5556383589f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.132+0000 7f66c0e00640 1 -- 172.21.15.103:0/2719295507 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563827aae0 con 0x555638358f30 2024-09-16T02:06:29.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.132+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2719295507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555638358f30 0x55563835b320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f66bc110730 tx=0x7f66a8009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb127a1507d2d626 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:29.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.132+0000 7f66ba000640 1 -- 172.21.15.103:0/673566455 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f66b0010ac0 con 0x7f66bc100720 2024-09-16T02:06:29.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.133+0000 7f66aea00640 1 -- 172.21.15.103:0/2719295507 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8011da0 con 0x555638358f30 2024-09-16T02:06:29.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.133+0000 7f66aea00640 1 -- 172.21.15.103:0/2719295507 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a8016070 con 0x555638358f30 2024-09-16T02:06:29.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.133+0000 7f66aea00640 1 -- 172.21.15.103:0/2719295507 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8012450 con 0x555638358f30 2024-09-16T02:06:29.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.134+0000 7f66c6e25740 1 -- 172.21.15.103:0/2719295507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555638358f30 msgr2=0x55563835b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.134+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2719295507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555638358f30 0x55563835b320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f66bc110730 tx=0x7f66a8009f50 comp rx=0 tx=0).stop 2024-09-16T02:06:29.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.134+0000 7f66c6e25740 1 -- 172.21.15.103:0/2719295507 shutdown_connections 2024-09-16T02:06:29.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.134+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2719295507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555638358f30 0x55563835b320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.134+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2719295507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834e670 0x5556383589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.134+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2719295507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563834dd50 0x55563834e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.134+0000 7f66c6e25740 1 -- 172.21.15.103:0/2719295507 >> 172.21.15.103:0/2719295507 conn(0x55563834a2e0 msgr2=0x55563834a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.135+0000 7f66c6e25740 1 -- 172.21.15.103:0/2719295507 shutdown_connections 2024-09-16T02:06:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.135+0000 7f66c6e25740 1 -- 172.21.15.103:0/2719295507 wait complete. 2024-09-16T02:06:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.136+0000 7f66ba000640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f66a40a32e0 0x7f66a40a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.136+0000 7f66c6e25740 1 -- start start 2024-09-16T02:06:29.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.136+0000 7f66c2200640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f66a40a32e0 0x7f66a40a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.136+0000 7f66c6e25740 1 Processor -- start 2024-09-16T02:06:29.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.137+0000 7f66ba000640 1 -- 172.21.15.103:0/673566455 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f66b00cff90 con 0x7f66bc100720 2024-09-16T02:06:29.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.137+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834dd50 0x555638356120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.137+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x555638356660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.137+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x555638365090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.137+0000 7f66c6e25740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55563835bba0 con 0x55563834dd50 2024-09-16T02:06:29.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.137+0000 7f66c6e25740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55563826f0f0 con 0x555638356bf0 2024-09-16T02:06:29.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.137+0000 7f66c6e25740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55563835bd20 con 0x55563834e670 2024-09-16T02:06:29.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.137+0000 7f66c6e25740 10 client.0 _ll_register_callbacks cb 0x5556382609e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:29.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.138+0000 7f66c0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x555638356660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.138+0000 7f66c0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x555638356660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:58174/0 (socket says 172.21.15.103:58174) 2024-09-16T02:06:29.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.138+0000 7f66c0e00640 1 -- 172.21.15.103:0/2526749240 learned_addr learned my addr 172.21.15.103:0/2526749240 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:29.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.138+0000 7f66c2200640 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x555638365090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.138+0000 7f66c2200640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f66a40a32e0 0x7f66a40a57a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f66b402ade0 tx=0x7f66b4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.138+0000 7f66c0e00640 1 -- 172.21.15.103:0/2526749240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 msgr2=0x555638365090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.138+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x555638365090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.138+0000 7f66c0e00640 1 -- 172.21.15.103:0/2526749240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834dd50 msgr2=0x555638356120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.138+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834dd50 0x555638356120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.138+0000 7f66c0e00640 1 -- 172.21.15.103:0/2526749240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556380253c0 con 0x55563834e670 2024-09-16T02:06:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.139+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x555638356660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f66a8003b40 tx=0x7f66a8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3f4bcd62162cd3a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.139+0000 7f66a3400640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:06:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.139+0000 7f66a3400640 1 -- 172.21.15.103:0/2526749240 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a802b060 con 0x55563834e670 2024-09-16T02:06:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.139+0000 7f66a3400640 1 -- 172.21.15.103:0/2526749240 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f66a8003f50 con 0x55563834e670 2024-09-16T02:06:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.139+0000 7f66c6e25740 1 -- 172.21.15.103:0/2526749240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a8016070 con 0x55563834e670 2024-09-16T02:06:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.140+0000 7f66a3400640 1 -- 172.21.15.103:0/2526749240 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f66a8004170 con 0x55563834e670 2024-09-16T02:06:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.140+0000 7f66c6e25740 10 client.25231 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.140+0000 7f66c6e25740 1 -- 172.21.15.103:0/2526749240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555638367fb0 con 0x55563834e670 2024-09-16T02:06:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.140+0000 7f66a3400640 1 -- 172.21.15.103:0/2526749240 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a8016070 con 0x55563834e670 2024-09-16T02:06:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.141+0000 7f66a3400640 1 -- 172.21.15.103:0/2526749240 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f66a801bb80 con 0x55563834e670 2024-09-16T02:06:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.141+0000 7f66c6e25740 10 client.25231 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.141+0000 7f66c6e25740 10 client.25231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.141+0000 7f66c6e25740 20 client.25231 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.141+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x555638369360 0x55563836b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.141+0000 7f66c6e25740 4 client.25231 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:06:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.141+0000 7f66c6e25740 1 -- 172.21.15.103:0/2526749240 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556382c1950 con 0x555638369360 2024-09-16T02:06:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.141+0000 7f66c1800640 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x555638369360 0x55563836b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.142+0000 7f66c1800640 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x555638369360 0x55563836b750 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5897dea1d2d4a24d server_cookie=80f14d54c054e7bb in_seq=0 out_seq=0 2024-09-16T02:06:29.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.142+0000 7f66a3400640 10 client.25231 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:29.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66a3400640 1 -- 172.21.15.103:0/2526749240 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556382c1950 con 0x555638369360 2024-09-16T02:06:29.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66a3400640 10 client.25231 handle_command_reply: tid=0 2024-09-16T02:06:29.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66c6e25740 1 client.25231 shutdown 2024-09-16T02:06:29.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66c6e25740 1 -- 172.21.15.103:0/2526749240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 msgr2=0x555638356660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x555638356660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f66a8003b40 tx=0x7f66a8005740 comp rx=0 tx=0).stop 2024-09-16T02:06:29.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66c6e25740 1 -- 172.21.15.103:0/2526749240 shutdown_connections 2024-09-16T02:06:29.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x555638369360 0x55563836b750 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x555638365090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x555638356660 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2526749240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834dd50 0x555638356120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66c6e25740 1 -- 172.21.15.103:0/2526749240 >> 172.21.15.103:0/2526749240 conn(0x55563834a2e0 msgr2=0x555638350040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:29.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.143+0000 7f66a3400640 0 client.25231 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:29.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.144+0000 7f66c6e25740 1 -- 172.21.15.103:0/2526749240 shutdown_connections 2024-09-16T02:06:29.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.144+0000 7f66c6e25740 1 -- 172.21.15.103:0/2526749240 wait complete. 2024-09-16T02:06:29.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.144+0000 7f66c6e25740 20 client.25231 trim_cache size 0 max 16384 2024-09-16T02:06:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.162+0000 7f66c6e25740 1 Processor -- start 2024-09-16T02:06:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.162+0000 7f66c6e25740 1 -- start start 2024-09-16T02:06:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834dd50 0x555638365000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834e670 0x55563836e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x55563836ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c6e25740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc1bede0 con 0x55563834e670 2024-09-16T02:06:29.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c6e25740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc1bc6b0 con 0x555638356bf0 2024-09-16T02:06:29.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c6e25740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc1ba500 con 0x55563834dd50 2024-09-16T02:06:29.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c1800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834dd50 0x555638365000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c1800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834dd50 0x555638365000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:58182/0 (socket says 172.21.15.103:58182) 2024-09-16T02:06:29.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c1800640 1 -- 172.21.15.103:0/2758650621 learned_addr learned my addr 172.21.15.103:0/2758650621 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:29.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834e670 0x55563836e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.163+0000 7f66c2200640 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x55563836ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.164+0000 7f66c1800640 1 -- 172.21.15.103:0/2758650621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 msgr2=0x55563836ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:29.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.164+0000 7f66c1800640 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x55563836ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.164+0000 7f66c1800640 1 -- 172.21.15.103:0/2758650621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834e670 msgr2=0x55563836e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:29.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.164+0000 7f66c1800640 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834e670 0x55563836e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.164+0000 7f66c1800640 1 -- 172.21.15.103:0/2758650621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555638365540 con 0x55563834dd50 2024-09-16T02:06:29.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.164+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834e670 0x55563836e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:06:29.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.164+0000 7f66c2200640 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x55563836ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:06:29.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.165+0000 7f66c1800640 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834dd50 0x555638365000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66b009c070 tx=0x7f66b00bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa5b3ae60d868b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:29.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.165+0000 7f66ae000640 1 -- 172.21.15.103:0/2758650621 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00a4fe0 con 0x55563834dd50 2024-09-16T02:06:29.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.165+0000 7f66ae000640 1 -- 172.21.15.103:0/2758650621 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66b009a670 con 0x55563834dd50 2024-09-16T02:06:29.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.165+0000 7f66ae000640 1 -- 172.21.15.103:0/2758650621 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0018d90 con 0x55563834dd50 2024-09-16T02:06:29.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.166+0000 7f66c6e25740 1 -- 172.21.15.103:0/2758650621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834dd50 msgr2=0x555638365000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.166+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834dd50 0x555638365000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66b009c070 tx=0x7f66b00bc040 comp rx=0 tx=0).stop 2024-09-16T02:06:29.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.166+0000 7f66c6e25740 1 -- 172.21.15.103:0/2758650621 shutdown_connections 2024-09-16T02:06:29.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.166+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x55563836ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.166+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834e670 0x55563836e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.166+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2758650621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834dd50 0x555638365000 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.166+0000 7f66c6e25740 1 -- 172.21.15.103:0/2758650621 >> 172.21.15.103:0/2758650621 conn(0x55563834a2e0 msgr2=0x555638354a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:29.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.166+0000 7f66c6e25740 1 -- 172.21.15.103:0/2758650621 shutdown_connections 2024-09-16T02:06:29.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.166+0000 7f66c6e25740 1 -- 172.21.15.103:0/2758650621 wait complete. 2024-09-16T02:06:29.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.167+0000 7f66c6e25740 1 -- start start 2024-09-16T02:06:29.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.167+0000 7f66c6e25740 1 Processor -- start 2024-09-16T02:06:29.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.167+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834dd50 0x55563836c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.167+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x55563836cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c6e25740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x5556383d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c6e25740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc1bab70 con 0x55563834dd50 2024-09-16T02:06:29.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c6e25740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc1bbfe0 con 0x555638356bf0 2024-09-16T02:06:29.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c6e25740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc1bb240 con 0x55563834e670 2024-09-16T02:06:29.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c6e25740 10 client.0 _ll_register_callbacks cb 0x7f66a8011f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:29.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x55563836cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x55563836cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:58196/0 (socket says 172.21.15.103:58196) 2024-09-16T02:06:29.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c0e00640 1 -- 172.21.15.103:0/2656372061 learned_addr learned my addr 172.21.15.103:0/2656372061 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:29.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c2200640 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x5556383d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c1800640 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834dd50 0x55563836c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c0e00640 1 -- 172.21.15.103:0/2656372061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 msgr2=0x5556383d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.168+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x5556383d2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.169+0000 7f66c0e00640 1 -- 172.21.15.103:0/2656372061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834dd50 msgr2=0x55563836c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.169+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834dd50 0x55563836c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.169+0000 7f66c0e00640 1 -- 172.21.15.103:0/2656372061 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563836d640 con 0x55563834e670 2024-09-16T02:06:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.169+0000 7f66c1800640 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834dd50 0x55563836c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:06:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.169+0000 7f66c0e00640 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x55563836cd90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66a8000c00 tx=0x7f66a80126a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9077f50b253b957 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.169+0000 7f66a3400640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:06:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.169+0000 7f66a3400640 1 -- 172.21.15.103:0/2656372061 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a802b050 con 0x55563834e670 2024-09-16T02:06:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.169+0000 7f66a3400640 1 -- 172.21.15.103:0/2656372061 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f66a8003800 con 0x55563834e670 2024-09-16T02:06:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.169+0000 7f66c6e25740 1 -- 172.21.15.103:0/2656372061 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b009a670 con 0x55563834e670 2024-09-16T02:06:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.170+0000 7f66a3400640 1 -- 172.21.15.103:0/2656372061 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f66a8003ba0 con 0x55563834e670 2024-09-16T02:06:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.170+0000 7f66c6e25740 10 client.25237 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.170+0000 7f66c6e25740 1 -- 172.21.15.103:0/2656372061 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5556383d42d0 con 0x55563834e670 2024-09-16T02:06:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.170+0000 7f66a3400640 1 -- 172.21.15.103:0/2656372061 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66b009a670 con 0x55563834e670 2024-09-16T02:06:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.170+0000 7f66a3400640 1 -- 172.21.15.103:0/2656372061 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f66a8004680 con 0x55563834e670 2024-09-16T02:06:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.170+0000 7f66c6e25740 10 client.25237 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.170+0000 7f66c6e25740 10 client.25237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.170+0000 7f66c6e25740 20 client.25237 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.171+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5556383d4a20 0x5556383d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.171+0000 7f66c6e25740 4 client.25237 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:06:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.171+0000 7f66c6e25740 1 -- 172.21.15.103:0/2656372061 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556382bbe60 con 0x5556383d4a20 2024-09-16T02:06:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.171+0000 7f66c1800640 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5556383d4a20 0x5556383d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66c1800640 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5556383d4a20 0x5556383d6e40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e7793af3f3fb30 server_cookie=17d7b0b543c64f13 in_seq=0 out_seq=0 2024-09-16T02:06:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66a3400640 10 client.25237 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66a3400640 1 -- 172.21.15.103:0/2656372061 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5556382bbe60 con 0x5556383d4a20 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66a3400640 10 client.25237 handle_command_reply: tid=0 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66c6e25740 1 client.25237 shutdown 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66c6e25740 1 -- 172.21.15.103:0/2656372061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 msgr2=0x55563836cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x55563836cd90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66a8000c00 tx=0x7f66a80126a0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66c6e25740 1 -- 172.21.15.103:0/2656372061 shutdown_connections 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5556383d4a20 0x5556383d6e40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555638356bf0 0x5556383d2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55563834e670 0x55563836cd90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66c6e25740 1 --2- 172.21.15.103:0/2656372061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55563834dd50 0x55563836c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66a3400640 0 client.25237 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.172+0000 7f66c6e25740 1 -- 172.21.15.103:0/2656372061 >> 172.21.15.103:0/2656372061 conn(0x55563834a2e0 msgr2=0x55563834f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.173+0000 7f66c6e25740 1 -- 172.21.15.103:0/2656372061 shutdown_connections 2024-09-16T02:06:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.173+0000 7f66c6e25740 1 -- 172.21.15.103:0/2656372061 wait complete. 2024-09-16T02:06:29.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.173+0000 7f66c6e25740 20 client.25237 trim_cache size 0 max 16384 2024-09-16T02:06:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:06:29.178 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T02:06:29.178 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-16T02:06:29.178 INFO:teuthology.orchestra.run.smithi103.stdout: "da49be25-2a4c-4d26-b93b-9a5eca4137d0": { 2024-09-16T02:06:29.178 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-16T02:06:29.179 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "da49be25-2a4c-4d26-b93b-9a5eca4137d0", 2024-09-16T02:06:29.179 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-16T02:06:29.179 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:06:29.179 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:06:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:06:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.174+0000 7f66c2c00640 1 -- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f66a40a32e0 msgr2=0x7f66a40a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.174+0000 7f66c2c00640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f66a40a32e0 0x7f66a40a57a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f66b402ade0 tx=0x7f66b4004a40 comp rx=0 tx=0).stop 2024-09-16T02:06:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.174+0000 7f66c2c00640 1 -- 172.21.15.103:0/673566455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66bc100720 msgr2=0x7f66bc107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.174+0000 7f66c2c00640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66bc100720 0x7f66bc107390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f66b000e9f0 tx=0x7f66b000eeb0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.174+0000 7f66c2c00640 1 -- 172.21.15.103:0/673566455 shutdown_connections 2024-09-16T02:06:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.174+0000 7f66c2c00640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f66a40a32e0 0x7f66a40a57a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.174+0000 7f66c2c00640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66bc10e2d0 0x7f66bc102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.174+0000 7f66c2c00640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66bc1010d0 0x7f66bc1078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.174+0000 7f66c2c00640 1 --2- 172.21.15.103:0/673566455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66bc100720 0x7f66bc107390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.174+0000 7f66c2c00640 1 -- 172.21.15.103:0/673566455 >> 172.21.15.103:0/673566455 conn(0x7f66bc0fc610 msgr2=0x7f66bc0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:29.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.175+0000 7f66c2c00640 1 -- 172.21.15.103:0/673566455 shutdown_connections 2024-09-16T02:06:29.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:06:29.175+0000 7f66c2c00640 1 -- 172.21.15.103:0/673566455 wait complete. 2024-09-16T02:06:29.944 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:29 smithi103 ceph-mon[30398]: pgmap v371: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-16T02:06:29.988 INFO:tasks.cephfs.filesystem:scrub status for tag:da49be25-2a4c-4d26-b93b-9a5eca4137d0 - {'path': '/', 'tag': 'da49be25-2a4c-4d26-b93b-9a5eca4137d0', 'options': 'recursive,force'} 2024-09-16T02:06:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:29 smithi136 ceph-mon[35323]: pgmap v371: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-16T02:06:30.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:29 smithi172 ceph-mon[35340]: pgmap v371: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-16T02:06:31.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:30 smithi103 ceph-mon[30398]: scrub summary: idle 2024-09-16T02:06:31.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:30 smithi136 ceph-mon[35323]: scrub summary: idle 2024-09-16T02:06:31.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:30 smithi172 ceph-mon[35340]: scrub summary: idle 2024-09-16T02:06:32.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:31 smithi103 ceph-mon[30398]: pgmap v372: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-16T02:06:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:31 smithi136 ceph-mon[35323]: pgmap v372: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-16T02:06:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:31 smithi172 ceph-mon[35340]: pgmap v372: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-16T02:06:34.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:33 smithi103 ceph-mon[30398]: pgmap v373: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-16T02:06:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:33 smithi136 ceph-mon[35323]: pgmap v373: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-16T02:06:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.040+0000 7fba26600640 1 -- 172.21.15.103:0/2839257805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba20074680 msgr2=0x7fba20074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.040+0000 7fba26600640 1 --2- 172.21.15.103:0/2839257805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba20074680 0x7fba20074a80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fba14009900 tx=0x7fba1402a400 comp rx=0 tx=0).stop 2024-09-16T02:06:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.040+0000 7fba26600640 1 -- 172.21.15.103:0/2839257805 shutdown_connections 2024-09-16T02:06:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.040+0000 7fba26600640 1 --2- 172.21.15.103:0/2839257805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba20105340 0x7fba20110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.040+0000 7fba26600640 1 --2- 172.21.15.103:0/2839257805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba20074fc0 0x7fba2010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.040+0000 7fba26600640 1 --2- 172.21.15.103:0/2839257805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba20074680 0x7fba20074a80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.040+0000 7fba26600640 1 -- 172.21.15.103:0/2839257805 >> 172.21.15.103:0/2839257805 conn(0x7fba200fd0d0 msgr2=0x7fba200ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.042+0000 7fba26600640 1 -- 172.21.15.103:0/2839257805 shutdown_connections 2024-09-16T02:06:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.042+0000 7fba26600640 1 -- 172.21.15.103:0/2839257805 wait complete. 2024-09-16T02:06:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.043+0000 7fba26600640 1 Processor -- start 2024-09-16T02:06:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.043+0000 7fba26600640 1 -- start start 2024-09-16T02:06:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba26600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba20074680 0x7fba2019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba26600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba20074fc0 0x7fba2019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba26600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba201a3940 0x7fba201a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba26600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba20113d90 con 0x7fba201a3940 2024-09-16T02:06:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba26600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba20113c10 con 0x7fba20074fc0 2024-09-16T02:06:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba26600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fba20113f10 con 0x7fba20074680 2024-09-16T02:06:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba25200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba20074680 0x7fba2019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba25200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba20074680 0x7fba2019e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:58212/0 (socket says 172.21.15.103:58212) 2024-09-16T02:06:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba25200640 1 -- 172.21.15.103:0/3118366747 learned_addr learned my addr 172.21.15.103:0/3118366747 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba25c00640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba201a3940 0x7fba201a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.044+0000 7fba1fe00640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba20074fc0 0x7fba2019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.076 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:33 smithi172 ceph-mon[35340]: pgmap v373: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.045+0000 7fba25200640 1 -- 172.21.15.103:0/3118366747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba20074fc0 msgr2=0x7fba2019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.045+0000 7fba25200640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba20074fc0 0x7fba2019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.045+0000 7fba25200640 1 -- 172.21.15.103:0/3118366747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba201a3940 msgr2=0x7fba201a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.045+0000 7fba25200640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba201a3940 0x7fba201a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.045+0000 7fba25200640 1 -- 172.21.15.103:0/3118366747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba201a63a0 con 0x7fba20074680 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.045+0000 7fba25200640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba20074680 0x7fba2019e3a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba14031440 tx=0x7fba14032c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.046+0000 7fba1d600640 1 -- 172.21.15.103:0/3118366747 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1403f030 con 0x7fba20074680 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.046+0000 7fba1d600640 1 -- 172.21.15.103:0/3118366747 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba14003ad0 con 0x7fba20074680 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.046+0000 7fba26600640 1 -- 172.21.15.103:0/3118366747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba201a6630 con 0x7fba20074680 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.046+0000 7fba1d600640 1 -- 172.21.15.103:0/3118366747 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14035070 con 0x7fba20074680 2024-09-16T02:06:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.046+0000 7fba26600640 1 -- 172.21.15.103:0/3118366747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba201a6ad0 con 0x7fba20074680 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.050+0000 7fba2aa19740 1 Processor -- start 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.051+0000 7fba2aa19740 1 -- start start 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.051+0000 7fba1d600640 1 -- 172.21.15.103:0/3118366747 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fba14042030 con 0x7fba20074680 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.051+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2bec2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.051+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2bec7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.051+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 0x563d2beca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.051+0000 7fba2aa19740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d2be27100 con 0x563d2bec2ba0 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.051+0000 7fba2aa19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563d2bdcf1b0 con 0x563d2bec2260 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.051+0000 7fba2aa19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563d2be230c0 con 0x563d2bec7c30 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba25c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2bec2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba25c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2bec2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:48534/0 (socket says 172.21.15.103:48534) 2024-09-16T02:06:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba25c00640 1 -- 172.21.15.103:0/3921965937 learned_addr learned my addr 172.21.15.103:0/3921965937 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba1fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 0x563d2beca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba25200640 1 --2- 172.21.15.103:0/3921965937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2bec7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba25c00640 1 -- 172.21.15.103:0/3921965937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 msgr2=0x563d2beca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba25c00640 1 --2- 172.21.15.103:0/3921965937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 0x563d2beca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba25c00640 1 -- 172.21.15.103:0/3921965937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 msgr2=0x563d2bec7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba25c00640 1 --2- 172.21.15.103:0/3921965937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2bec7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba25c00640 1 -- 172.21.15.103:0/3921965937 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2bb3a210 con 0x563d2bec2260 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba1fe00640 1 --2- 172.21.15.103:0/3921965937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 0x563d2beca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.052+0000 7fba25200640 1 --2- 172.21.15.103:0/3921965937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2bec7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.053+0000 7fba25c00640 1 --2- 172.21.15.103:0/3921965937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2bec2660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fba18009fd0 tx=0x7fba180118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc5808981e042a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.053+0000 7fba12000640 1 -- 172.21.15.103:0/3921965937 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18004d20 con 0x563d2bec2260 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.053+0000 7fba12000640 1 -- 172.21.15.103:0/3921965937 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba18004050 con 0x563d2bec2260 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.053+0000 7fba12000640 1 -- 172.21.15.103:0/3921965937 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18005220 con 0x563d2bec2260 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.053+0000 7fba1d600640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba080a3320 0x7fba080a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.054+0000 7fba1d600640 1 -- 172.21.15.103:0/3118366747 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fba140f0020 con 0x7fba20074680 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.054+0000 7fba25200640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba080a3320 0x7fba080a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.054+0000 7fba2aa19740 1 -- 172.21.15.103:0/3921965937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 msgr2=0x563d2bec2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.054+0000 7fba2aa19740 1 --2- 172.21.15.103:0/3921965937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2bec2660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fba18009fd0 tx=0x7fba180118d0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.054+0000 7fba2aa19740 1 -- 172.21.15.103:0/3921965937 shutdown_connections 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.054+0000 7fba2aa19740 1 --2- 172.21.15.103:0/3921965937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 0x563d2beca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.054+0000 7fba2aa19740 1 --2- 172.21.15.103:0/3921965937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2bec7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.054+0000 7fba2aa19740 1 --2- 172.21.15.103:0/3921965937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2bec2660 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.055+0000 7fba2aa19740 1 -- 172.21.15.103:0/3921965937 >> 172.21.15.103:0/3921965937 conn(0x563d2bebd8d0 msgr2=0x563d2bebdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.055+0000 7fba2aa19740 1 -- 172.21.15.103:0/3921965937 shutdown_connections 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.055+0000 7fba25200640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba080a3320 0x7fba080a57e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x563d2bec35d0 tx=0x7fba14009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.056+0000 7fba2aa19740 1 -- 172.21.15.103:0/3921965937 wait complete. 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.056+0000 7fba2aa19740 1 -- start start 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.056+0000 7fba2aa19740 1 Processor -- start 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.057+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 0x563d2becc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.057+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2becc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.057+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec7c30 0x563d2bed3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.057+0000 7fba2aa19740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d2becac00 con 0x563d2bec2ba0 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.057+0000 7fba2aa19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563d2bdb0090 con 0x563d2bec7c30 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.057+0000 7fba2aa19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563d2becad80 con 0x563d2bec2260 2024-09-16T02:06:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.057+0000 7fba2aa19740 10 client.0 _ll_register_callbacks cb 0x563d2be04f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.058+0000 7fba25c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec7c30 0x563d2bed3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.058+0000 7fba25c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 0x563d2becc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.058+0000 7fba25c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 0x563d2becc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:58234/0 (socket says 172.21.15.103:58234) 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.058+0000 7fba25c00640 1 -- 172.21.15.103:0/901916959 learned_addr learned my addr 172.21.15.103:0/901916959 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.058+0000 7fba1fe00640 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2becc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.058+0000 7fba25c00640 1 -- 172.21.15.103:0/901916959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec7c30 msgr2=0x563d2bed3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.058+0000 7fba25c00640 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec7c30 0x563d2bed3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.058+0000 7fba25c00640 1 -- 172.21.15.103:0/901916959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 msgr2=0x563d2becc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.058+0000 7fba25c00640 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2becc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.059+0000 7fba25c00640 1 -- 172.21.15.103:0/901916959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d2bc06620 con 0x563d2bec2260 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.059+0000 7fba1fe00640 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2becc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.059+0000 7fba25c00640 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 0x563d2becc3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba18011dd0 tx=0x7fba18005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d8373208beaf388 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.060+0000 7fba06a00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.060+0000 7fba06a00640 1 -- 172.21.15.103:0/901916959 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1800f6f0 con 0x563d2bec2260 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.060+0000 7fba06a00640 1 -- 172.21.15.103:0/901916959 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fba1800f890 con 0x563d2bec2260 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.060+0000 7fba2aa19740 1 -- 172.21.15.103:0/901916959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba18004050 con 0x563d2bec2260 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.060+0000 7fba06a00640 1 -- 172.21.15.103:0/901916959 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fba1800fa90 con 0x563d2bec2260 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.060+0000 7fba2aa19740 10 client.25249 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.060+0000 7fba2aa19740 1 -- 172.21.15.103:0/901916959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563d2bed66a0 con 0x563d2bec2260 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.060+0000 7fba06a00640 1 -- 172.21.15.103:0/901916959 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba18004050 con 0x563d2bec2260 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.061+0000 7fba06a00640 1 -- 172.21.15.103:0/901916959 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fba18025040 con 0x563d2bec2260 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.061+0000 7fba2aa19740 10 client.25249 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.061+0000 7fba2aa19740 10 client.25249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.061+0000 7fba2aa19740 20 client.25249 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.061+0000 7fba2aa19740 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x563d2bed7ba0 0x563d2bed9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.061+0000 7fba2aa19740 4 client.25249 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.061+0000 7fba2aa19740 1 -- 172.21.15.103:0/901916959 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d2be34830 con 0x563d2bed7ba0 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.062+0000 7fba1fe00640 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x563d2bed7ba0 0x563d2bed9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.062+0000 7fba1fe00640 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x563d2bed7ba0 0x563d2bed9f90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b923c0864fdd457a server_cookie=c4ad0d5967ddc0f4 in_seq=0 out_seq=0 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.062+0000 7fba06a00640 10 client.25249 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.063+0000 7fba06a00640 1 -- 172.21.15.103:0/901916959 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d2be34830 con 0x563d2bed7ba0 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.063+0000 7fba06a00640 10 client.25249 handle_command_reply: tid=0 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.063+0000 7fba2aa19740 1 client.25249 shutdown 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 1 -- 172.21.15.103:0/901916959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 msgr2=0x563d2becc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 0x563d2becc3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba18011dd0 tx=0x7fba18005fd0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 1 -- 172.21.15.103:0/901916959 shutdown_connections 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x563d2bed7ba0 0x563d2bed9f90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec7c30 0x563d2bed3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2becc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 1 --2- 172.21.15.103:0/901916959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 0x563d2becc3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 1 -- 172.21.15.103:0/901916959 >> 172.21.15.103:0/901916959 conn(0x563d2bebd8d0 msgr2=0x563d2bebf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba06a00640 0 client.25249 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 1 -- 172.21.15.103:0/901916959 shutdown_connections 2024-09-16T02:06:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 1 -- 172.21.15.103:0/901916959 wait complete. 2024-09-16T02:06:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.064+0000 7fba2aa19740 20 client.25249 trim_cache size 0 max 16384 2024-09-16T02:06:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.094+0000 7fba2aa19740 1 Processor -- start 2024-09-16T02:06:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.094+0000 7fba2aa19740 1 -- start start 2024-09-16T02:06:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.094+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2becc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.094+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2bed38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.094+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 0x563d2becc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.094+0000 7fba2aa19740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba201b1fc0 con 0x563d2bec2ba0 2024-09-16T02:06:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.094+0000 7fba2aa19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba201af890 con 0x563d2bec2260 2024-09-16T02:06:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.094+0000 7fba2aa19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fba201ad6e0 con 0x563d2bec7c30 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba25c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2becc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba1fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2bed38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba25c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2becc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:48550/0 (socket says 172.21.15.103:48550) 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba25c00640 1 -- 172.21.15.103:0/3479638542 learned_addr learned my addr 172.21.15.103:0/3479638542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba25c00640 1 --2- 172.21.15.103:0/3479638542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 0x563d2becc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba25c00640 1 -- 172.21.15.103:0/3479638542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 msgr2=0x563d2becc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba25c00640 1 --2- 172.21.15.103:0/3479638542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 0x563d2becc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba25c00640 1 -- 172.21.15.103:0/3479638542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 msgr2=0x563d2bed38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba25c00640 1 --2- 172.21.15.103:0/3479638542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2bed38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba25c00640 1 -- 172.21.15.103:0/3479638542 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2bec7020 con 0x563d2bec2260 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.095+0000 7fba25c00640 1 --2- 172.21.15.103:0/3479638542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2becc170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba180041d0 tx=0x7fba18014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86ee5066f4447ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.096+0000 7fba11600640 1 -- 172.21.15.103:0/3479638542 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18019430 con 0x563d2bec2260 2024-09-16T02:06:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.096+0000 7fba11600640 1 -- 172.21.15.103:0/3479638542 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1802d070 con 0x563d2bec2260 2024-09-16T02:06:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.096+0000 7fba11600640 1 -- 172.21.15.103:0/3479638542 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1800f6f0 con 0x563d2bec2260 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.097+0000 7fba2aa19740 1 -- 172.21.15.103:0/3479638542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 msgr2=0x563d2becc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.097+0000 7fba2aa19740 1 --2- 172.21.15.103:0/3479638542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2becc170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba180041d0 tx=0x7fba18014040 comp rx=0 tx=0).stop 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.097+0000 7fba2aa19740 1 -- 172.21.15.103:0/3479638542 shutdown_connections 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.097+0000 7fba2aa19740 1 --2- 172.21.15.103:0/3479638542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec7c30 0x563d2becc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.097+0000 7fba2aa19740 1 --2- 172.21.15.103:0/3479638542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec2ba0 0x563d2bed38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.097+0000 7fba2aa19740 1 --2- 172.21.15.103:0/3479638542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2260 0x563d2becc170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.097+0000 7fba2aa19740 1 -- 172.21.15.103:0/3479638542 >> 172.21.15.103:0/3479638542 conn(0x563d2bebd8d0 msgr2=0x563d2bec3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.097+0000 7fba2aa19740 1 -- 172.21.15.103:0/3479638542 shutdown_connections 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.097+0000 7fba2aa19740 1 -- 172.21.15.103:0/3479638542 wait complete. 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.098+0000 7fba2aa19740 1 -- start start 2024-09-16T02:06:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.098+0000 7fba2aa19740 1 Processor -- start 2024-09-16T02:06:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.098+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 0x563d2bed3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.098+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2ba0 0x563d2bed01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.098+0000 7fba2aa19740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec7c30 0x563d2bed0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba2aa19740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba201add50 con 0x563d2bec7c30 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba2aa19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba201af1c0 con 0x563d2bec2ba0 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba2aa19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fba201ae420 con 0x563d2bec2260 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba2aa19740 10 client.0 _ll_register_callbacks cb 0x7fba18009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba1fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2ba0 0x563d2bed01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba1fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2ba0 0x563d2bed01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:48552/0 (socket says 172.21.15.103:48552) 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba1fe00640 1 -- 172.21.15.103:0/823237196 learned_addr learned my addr 172.21.15.103:0/823237196 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba25c00640 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 0x563d2bed3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba25c00640 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec7c30 0x563d2bed0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba1fe00640 1 -- 172.21.15.103:0/823237196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 msgr2=0x563d2bed3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba1fe00640 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 0x563d2bed3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba1fe00640 1 -- 172.21.15.103:0/823237196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec7c30 msgr2=0x563d2bed0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba1fe00640 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec7c30 0x563d2bed0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba1fe00640 1 -- 172.21.15.103:0/823237196 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d2bf40810 con 0x563d2bec2ba0 2024-09-16T02:06:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.099+0000 7fba25c00640 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec7c30 0x563d2bed0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:06:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.100+0000 7fba1fe00640 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2ba0 0x563d2bed01c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x563d2beda7f0 tx=0x7fba0c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ad196c8c0e980c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.100+0000 7fba06a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:06:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.100+0000 7fba06a00640 1 -- 172.21.15.103:0/823237196 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c010430 con 0x563d2bec2ba0 2024-09-16T02:06:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.100+0000 7fba06a00640 1 -- 172.21.15.103:0/823237196 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fba0c0105d0 con 0x563d2bec2ba0 2024-09-16T02:06:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.100+0000 7fba2aa19740 1 -- 172.21.15.103:0/823237196 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1802d070 con 0x563d2bec2ba0 2024-09-16T02:06:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.101+0000 7fba06a00640 1 -- 172.21.15.103:0/823237196 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fba0c02e050 con 0x563d2bec2ba0 2024-09-16T02:06:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.101+0000 7fba2aa19740 10 client.25274 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.101+0000 7fba2aa19740 1 -- 172.21.15.103:0/823237196 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563d2bf41b40 con 0x563d2bec2ba0 2024-09-16T02:06:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.101+0000 7fba06a00640 1 -- 172.21.15.103:0/823237196 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1802d070 con 0x563d2bec2ba0 2024-09-16T02:06:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.102+0000 7fba06a00640 1 -- 172.21.15.103:0/823237196 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fba0c0107d0 con 0x563d2bec2ba0 2024-09-16T02:06:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.102+0000 7fba2aa19740 10 client.25274 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.102+0000 7fba2aa19740 10 client.25274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.102+0000 7fba2aa19740 20 client.25274 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.102+0000 7fba2aa19740 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x563d2bf42420 0x563d2bf44840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.102+0000 7fba2aa19740 4 client.25274 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T02:06:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.102+0000 7fba2aa19740 1 -- 172.21.15.103:0/823237196 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d2be2eb80 con 0x563d2bf42420 2024-09-16T02:06:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.103+0000 7fba25c00640 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x563d2bf42420 0x563d2bf44840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.104+0000 7fba25c00640 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x563d2bf42420 0x563d2bf44840 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3318eb5f85a69fd1 server_cookie=1e797fb3fb7866f1 in_seq=0 out_seq=0 2024-09-16T02:06:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.104+0000 7fba06a00640 10 client.25274 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.107+0000 7fba06a00640 1 -- 172.21.15.103:0/823237196 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d2be2eb80 con 0x563d2bf42420 2024-09-16T02:06:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.107+0000 7fba06a00640 10 client.25274 handle_command_reply: tid=0 2024-09-16T02:06:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.107+0000 7fba2aa19740 1 client.25274 shutdown 2024-09-16T02:06:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba2aa19740 1 -- 172.21.15.103:0/823237196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2ba0 msgr2=0x563d2bed01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba2aa19740 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2ba0 0x563d2bed01c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x563d2beda7f0 tx=0x7fba0c009b50 comp rx=0 tx=0).stop 2024-09-16T02:06:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba2aa19740 1 -- 172.21.15.103:0/823237196 shutdown_connections 2024-09-16T02:06:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba2aa19740 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x563d2bf42420 0x563d2bf44840 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba2aa19740 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d2bec7c30 0x563d2bed0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba2aa19740 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d2bec2ba0 0x563d2bed01c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba2aa19740 1 --2- 172.21.15.103:0/823237196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d2bec2260 0x563d2bed3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba2aa19740 1 -- 172.21.15.103:0/823237196 >> 172.21.15.103:0/823237196 conn(0x563d2bebd8d0 msgr2=0x563d2becc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba06a00640 0 client.25274 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba2aa19740 1 -- 172.21.15.103:0/823237196 shutdown_connections 2024-09-16T02:06:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.108+0000 7fba2aa19740 1 -- 172.21.15.103:0/823237196 wait complete. 2024-09-16T02:06:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.109+0000 7fba2aa19740 20 client.25274 trim_cache size 0 max 16384 2024-09-16T02:06:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-16T02:06:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-16T02:06:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-16T02:06:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-16T02:06:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.109+0000 7fba26600640 1 -- 172.21.15.103:0/3118366747 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba080a3320 msgr2=0x7fba080a57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.109+0000 7fba26600640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba080a3320 0x7fba080a57e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x563d2bec35d0 tx=0x7fba14009260 comp rx=0 tx=0).stop 2024-09-16T02:06:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.109+0000 7fba26600640 1 -- 172.21.15.103:0/3118366747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba20074680 msgr2=0x7fba2019e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.109+0000 7fba26600640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba20074680 0x7fba2019e3a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba14031440 tx=0x7fba14032c60 comp rx=0 tx=0).stop 2024-09-16T02:06:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.111+0000 7fba26600640 1 -- 172.21.15.103:0/3118366747 shutdown_connections 2024-09-16T02:06:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.111+0000 7fba26600640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fba080a3320 0x7fba080a57e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.111+0000 7fba26600640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba201a3940 0x7fba201a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.111+0000 7fba26600640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba20074fc0 0x7fba2019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.111+0000 7fba26600640 1 --2- 172.21.15.103:0/3118366747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba20074680 0x7fba2019e3a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.111+0000 7fba26600640 1 -- 172.21.15.103:0/3118366747 >> 172.21.15.103:0/3118366747 conn(0x7fba200fd0d0 msgr2=0x7fba201100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.111+0000 7fba26600640 1 -- 172.21.15.103:0/3118366747 shutdown_connections 2024-09-16T02:06:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:34.111+0000 7fba26600640 1 -- 172.21.15.103:0/3118366747 wait complete. 2024-09-16T02:06:36.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:35 smithi103 ceph-mon[30398]: pgmap v374: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-16T02:06:36.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:35 smithi136 ceph-mon[35323]: pgmap v374: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-16T02:06:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:35 smithi172 ceph-mon[35340]: pgmap v374: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-16T02:06:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:37 smithi103 ceph-mon[30398]: pgmap v375: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-16T02:06:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:37 smithi136 ceph-mon[35323]: pgmap v375: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-16T02:06:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:37 smithi172 ceph-mon[35340]: pgmap v375: 97 pgs: 97 active+clean; 672 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-16T02:06:40.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:39 smithi103 ceph-mon[30398]: pgmap v376: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-16T02:06:40.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:39 smithi136 ceph-mon[35323]: pgmap v376: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-16T02:06:40.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:39 smithi172 ceph-mon[35340]: pgmap v376: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-16T02:06:42.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:41 smithi103 ceph-mon[30398]: pgmap v377: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:06:42.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:41 smithi136 ceph-mon[35323]: pgmap v377: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:06:42.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:41 smithi172 ceph-mon[35340]: pgmap v377: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:06:44.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:43 smithi103 ceph-mon[30398]: pgmap v378: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T02:06:44.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:43 smithi136 ceph-mon[35323]: pgmap v378: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T02:06:44.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:43 smithi172 ceph-mon[35340]: pgmap v378: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T02:06:46.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:45 smithi103 ceph-mon[30398]: pgmap v379: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T02:06:46.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:45 smithi136 ceph-mon[35323]: pgmap v379: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T02:06:46.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:45 smithi172 ceph-mon[35340]: pgmap v379: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T02:06:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.235+0000 7f5911400640 1 -- 172.21.15.103:0/2976792435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590c074f40 msgr2=0x7f590c1097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.235+0000 7f5911400640 1 --2- 172.21.15.103:0/2976792435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590c074f40 0x7f590c1097d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009900 tx=0x7f58f402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.235+0000 7f5911400640 1 -- 172.21.15.103:0/2976792435 shutdown_connections 2024-09-16T02:06:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.235+0000 7f5911400640 1 --2- 172.21.15.103:0/2976792435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f590c109d10 0x7f590c10c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.235+0000 7f5911400640 1 --2- 172.21.15.103:0/2976792435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590c074f40 0x7f590c1097d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.235+0000 7f5911400640 1 --2- 172.21.15.103:0/2976792435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f590c074600 0x7f590c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.235+0000 7f5911400640 1 -- 172.21.15.103:0/2976792435 >> 172.21.15.103:0/2976792435 conn(0x7f590c0fd0d0 msgr2=0x7f590c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.236+0000 7f5911400640 1 -- 172.21.15.103:0/2976792435 shutdown_connections 2024-09-16T02:06:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.236+0000 7f5911400640 1 -- 172.21.15.103:0/2976792435 wait complete. 2024-09-16T02:06:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.236+0000 7f5911400640 1 Processor -- start 2024-09-16T02:06:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f5911400640 1 -- start start 2024-09-16T02:06:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f5911400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f590c074600 0x7f590c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f5911400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f590c074f40 0x7f590c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f5911400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590c109d10 0x7f590c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f5911400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f590c10f940 con 0x7f590c109d10 2024-09-16T02:06:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f5911400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f590c10f7c0 con 0x7f590c074f40 2024-09-16T02:06:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f5911400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f590c10fac0 con 0x7f590c074600 2024-09-16T02:06:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f5910a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590c109d10 0x7f590c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f590be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f590c074600 0x7f590c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f5903e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f590c074f40 0x7f590c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f590be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f590c074600 0x7f590c19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:59296/0 (socket says 172.21.15.103:59296) 2024-09-16T02:06:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f5903e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f590c074f40 0x7f590c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:36338/0 (socket says 172.21.15.103:36338) 2024-09-16T02:06:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.237+0000 7f590be00640 1 -- 172.21.15.103:0/3482859153 learned_addr learned my addr 172.21.15.103:0/3482859153 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5910a00640 1 -- 172.21.15.103:0/3482859153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f590c074600 msgr2=0x7f590c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5910a00640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f590c074600 0x7f590c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5910a00640 1 -- 172.21.15.103:0/3482859153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f590c074f40 msgr2=0x7f590c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5910a00640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f590c074f40 0x7f590c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5910a00640 1 -- 172.21.15.103:0/3482859153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590c1a1c20 con 0x7f590c109d10 2024-09-16T02:06:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f590be00640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f590c074600 0x7f590c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5910a00640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590c109d10 0x7f590c1a15b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f590400ec50 tx=0x7f590400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5909600640 1 -- 172.21.15.103:0/3482859153 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904003b70 con 0x7f590c109d10 2024-09-16T02:06:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5909600640 1 -- 172.21.15.103:0/3482859153 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5904003d10 con 0x7f590c109d10 2024-09-16T02:06:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5911400640 1 -- 172.21.15.103:0/3482859153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590c1a1f10 con 0x7f590c109d10 2024-09-16T02:06:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5909600640 1 -- 172.21.15.103:0/3482859153 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904010890 con 0x7f590c109d10 2024-09-16T02:06:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.238+0000 7f5911400640 1 -- 172.21.15.103:0/3482859153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590c1a23d0 con 0x7f590c109d10 2024-09-16T02:06:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.242+0000 7f591587a740 1 Processor -- start 2024-09-16T02:06:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.242+0000 7f5909600640 1 -- 172.21.15.103:0/3482859153 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f5904003fc0 con 0x7f590c109d10 2024-09-16T02:06:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f591587a740 1 -- start start 2024-09-16T02:06:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f591587a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f591587a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05bc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f591587a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05bf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f591587a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c7e051c100 con 0x55c7e05b7ba0 2024-09-16T02:06:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f591587a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c7e04c41b0 con 0x55c7e05bcc30 2024-09-16T02:06:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f591587a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c7e05180c0 con 0x55c7e05b7260 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f590be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f590be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05bc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59956/0 (socket says 172.21.15.103:59956) 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f590be00640 1 -- 172.21.15.103:0/1633507387 learned_addr learned my addr 172.21.15.103:0/1633507387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f5903e00640 1 --2- 172.21.15.103:0/1633507387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f5903e00640 1 --2- 172.21.15.103:0/1633507387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05bf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f590be00640 1 -- 172.21.15.103:0/1633507387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 msgr2=0x55c7e05b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f590be00640 1 --2- 172.21.15.103:0/1633507387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f590be00640 1 -- 172.21.15.103:0/1633507387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 msgr2=0x55c7e05bf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f590be00640 1 --2- 172.21.15.103:0/1633507387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05bf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.243+0000 7f590be00640 1 -- 172.21.15.103:0/1633507387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7e022f210 con 0x55c7e05b7ba0 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.244+0000 7f590be00640 1 --2- 172.21.15.103:0/1633507387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05bc660 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f58fc011630 tx=0x7f58fc011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92b9e5d58a97140 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.244+0000 7f5901200640 1 -- 172.21.15.103:0/1633507387 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc01e070 con 0x55c7e05b7ba0 2024-09-16T02:06:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.244+0000 7f5901200640 1 -- 172.21.15.103:0/1633507387 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58fc00fbd0 con 0x55c7e05b7ba0 2024-09-16T02:06:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.244+0000 7f5901200640 1 -- 172.21.15.103:0/1633507387 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc019d00 con 0x55c7e05b7ba0 2024-09-16T02:06:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.244+0000 7f5909600640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f58f00a3350 0x7f58f00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.245+0000 7f5909600640 1 -- 172.21.15.103:0/3482859153 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f59040d00b0 con 0x7f590c109d10 2024-09-16T02:06:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.245+0000 7f591587a740 1 -- 172.21.15.103:0/1633507387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 msgr2=0x55c7e05bc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.245+0000 7f591587a740 1 --2- 172.21.15.103:0/1633507387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05bc660 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f58fc011630 tx=0x7f58fc011af0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.245+0000 7f5903e00640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f58f00a3350 0x7f58f00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.245+0000 7f591587a740 1 -- 172.21.15.103:0/1633507387 shutdown_connections 2024-09-16T02:06:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.245+0000 7f591587a740 1 --2- 172.21.15.103:0/1633507387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05bf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.245+0000 7f591587a740 1 --2- 172.21.15.103:0/1633507387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05bc660 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.245+0000 7f591587a740 1 --2- 172.21.15.103:0/1633507387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.245+0000 7f591587a740 1 -- 172.21.15.103:0/1633507387 >> 172.21.15.103:0/1633507387 conn(0x55c7e05b28d0 msgr2=0x55c7e05b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.246+0000 7f591587a740 1 -- 172.21.15.103:0/1633507387 shutdown_connections 2024-09-16T02:06:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.246+0000 7f591587a740 1 -- 172.21.15.103:0/1633507387 wait complete. 2024-09-16T02:06:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.246+0000 7f5903e00640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f58f00a3350 0x7f58f00a5810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58f4031440 tx=0x7f58f40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.246+0000 7f591587a740 1 -- start start 2024-09-16T02:06:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.246+0000 7f591587a740 1 Processor -- start 2024-09-16T02:06:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f591587a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7260 0x55c7e05c1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f591587a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f591587a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f591587a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c7e05bfc00 con 0x55c7e05b7260 2024-09-16T02:06:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f591587a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c7e04a5090 con 0x55c7e05bcc30 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f591587a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c7e05bfd80 con 0x55c7e05b7ba0 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f591587a740 10 client.0 _ll_register_callbacks cb 0x55c7e04f9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f590be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7260 0x55c7e05c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f590be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7260 0x55c7e05c1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59972/0 (socket says 172.21.15.103:59972) 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f590be00640 1 -- 172.21.15.103:0/3813138005 learned_addr learned my addr 172.21.15.103:0/3813138005 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f5903e00640 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f5910a00640 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f590be00640 1 -- 172.21.15.103:0/3813138005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7ba0 msgr2=0x55c7e05c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.247+0000 7f590be00640 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f590be00640 1 -- 172.21.15.103:0/3813138005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 msgr2=0x55c7e05c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f590be00640 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f590be00640 1 -- 172.21.15.103:0/3813138005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7e02fb620 con 0x55c7e05b7260 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f590be00640 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7260 0x55c7e05c1440 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00f3c0 tx=0x7f58fc00ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd4edea894d1dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f58f9600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f58f9600640 1 -- 172.21.15.103:0/3813138005 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc026020 con 0x55c7e05b7260 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f58f9600640 1 -- 172.21.15.103:0/3813138005 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f58fc009ce0 con 0x55c7e05b7260 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f591587a740 1 -- 172.21.15.103:0/3813138005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58fc00fbd0 con 0x55c7e05b7260 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f58f9600640 1 -- 172.21.15.103:0/3813138005 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f58fc031050 con 0x55c7e05b7260 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f591587a740 10 client.15324 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.248+0000 7f591587a740 1 -- 172.21.15.103:0/3813138005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c7e05cb7b0 con 0x55c7e05b7260 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.249+0000 7f58f9600640 1 -- 172.21.15.103:0/3813138005 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58fc00fbd0 con 0x55c7e05b7260 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.249+0000 7f58f9600640 1 -- 172.21.15.103:0/3813138005 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f58fc021050 con 0x55c7e05b7260 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.249+0000 7f591587a740 10 client.15324 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.249+0000 7f591587a740 10 client.15324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.249+0000 7f591587a740 20 client.15324 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.249+0000 7f591587a740 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c7e05cccb0 0x55c7e05cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.249+0000 7f591587a740 4 client.15324 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.249+0000 7f591587a740 1 -- 172.21.15.103:0/3813138005 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7e0529830 con 0x55c7e05cccb0 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.249+0000 7f5903e00640 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c7e05cccb0 0x55c7e05cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.250+0000 7f5903e00640 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c7e05cccb0 0x55c7e05cf120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bbcf1d6945fffca server_cookie=7d8e0a33fa4a1d1 in_seq=0 out_seq=0 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.250+0000 7f58f9600640 10 client.15324 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.250+0000 7f58f9600640 1 -- 172.21.15.103:0/3813138005 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7e0529830 con 0x55c7e05cccb0 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.250+0000 7f58f9600640 10 client.15324 handle_command_reply: tid=0 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 client.15324 shutdown 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 -- 172.21.15.103:0/3813138005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7260 msgr2=0x55c7e05c1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7260 0x55c7e05c1440 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00f3c0 tx=0x7f58fc00ffb0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 -- 172.21.15.103:0/3813138005 shutdown_connections 2024-09-16T02:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c7e05cccb0 0x55c7e05cf120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 --2- 172.21.15.103:0/3813138005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7260 0x55c7e05c1440 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 -- 172.21.15.103:0/3813138005 >> 172.21.15.103:0/3813138005 conn(0x55c7e05b28d0 msgr2=0x55c7e05b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f58f9600640 0 client.15324 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 -- 172.21.15.103:0/3813138005 shutdown_connections 2024-09-16T02:06:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 1 -- 172.21.15.103:0/3813138005 wait complete. 2024-09-16T02:06:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.251+0000 7f591587a740 20 client.15324 trim_cache size 0 max 16384 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f591587a740 1 Processor -- start 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f591587a740 1 -- start start 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f591587a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f591587a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f591587a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f591587a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1adba0 con 0x55c7e05b7ba0 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f591587a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1ab470 con 0x55c7e05bcc30 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f591587a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1a92c0 con 0x55c7e05b7260 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f5903e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f5903e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59982/0 (socket says 172.21.15.103:59982) 2024-09-16T02:06:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f5903e00640 1 -- 172.21.15.103:0/1187373701 learned_addr learned my addr 172.21.15.103:0/1187373701 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.266+0000 7f590be00640 1 --2- 172.21.15.103:0/1187373701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f5910a00640 1 --2- 172.21.15.103:0/1187373701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f590be00640 1 -- 172.21.15.103:0/1187373701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 msgr2=0x55c7e05b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f590be00640 1 --2- 172.21.15.103:0/1187373701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f590be00640 1 -- 172.21.15.103:0/1187373701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 msgr2=0x55c7e05c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f590be00640 1 --2- 172.21.15.103:0/1187373701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f590be00640 1 -- 172.21.15.103:0/1187373701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7e05b8080 con 0x55c7e05b7260 2024-09-16T02:06:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f5903e00640 1 --2- 172.21.15.103:0/1187373701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f590be00640 1 --2- 172.21.15.103:0/1187373701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05b2cb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58fc004b70 tx=0x7f58fc004f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33fb203d08174140 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f58fbe00640 1 -- 172.21.15.103:0/1187373701 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc016070 con 0x55c7e05b7260 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f58fbe00640 1 -- 172.21.15.103:0/1187373701 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58fc005520 con 0x55c7e05b7260 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.267+0000 7f58fbe00640 1 -- 172.21.15.103:0/1187373701 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc026070 con 0x55c7e05b7260 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.268+0000 7f591587a740 1 -- 172.21.15.103:0/1187373701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 msgr2=0x55c7e05b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.268+0000 7f591587a740 1 --2- 172.21.15.103:0/1187373701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05b2cb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58fc004b70 tx=0x7f58fc004f10 comp rx=0 tx=0).stop 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.268+0000 7f591587a740 1 -- 172.21.15.103:0/1187373701 shutdown_connections 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.268+0000 7f591587a740 1 --2- 172.21.15.103:0/1187373701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05bcc30 0x55c7e05b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.268+0000 7f591587a740 1 --2- 172.21.15.103:0/1187373701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.268+0000 7f591587a740 1 --2- 172.21.15.103:0/1187373701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05b2cb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.268+0000 7f591587a740 1 -- 172.21.15.103:0/1187373701 >> 172.21.15.103:0/1187373701 conn(0x55c7e05b28d0 msgr2=0x55c7e05c11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.268+0000 7f591587a740 1 -- 172.21.15.103:0/1187373701 shutdown_connections 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.268+0000 7f591587a740 1 -- 172.21.15.103:0/1187373701 wait complete. 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.269+0000 7f591587a740 1 -- start start 2024-09-16T02:06:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.269+0000 7f591587a740 1 Processor -- start 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.269+0000 7f591587a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.269+0000 7f591587a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.269+0000 7f591587a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05bcc30 0x55c7e05c7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.269+0000 7f591587a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1a9930 con 0x55c7e05bcc30 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.269+0000 7f591587a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1aada0 con 0x55c7e05b7ba0 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.269+0000 7f591587a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1aa000 con 0x55c7e05b7260 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.269+0000 7f591587a740 10 client.0 _ll_register_callbacks cb 0x7f58fc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f5910a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05bcc30 0x55c7e05c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f590be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f5910a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05bcc30 0x55c7e05c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59998/0 (socket says 172.21.15.103:59998) 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f5903e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f5910a00640 1 -- 172.21.15.103:0/856423786 learned_addr learned my addr 172.21.15.103:0/856423786 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:06:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f590be00640 1 -- 172.21.15.103:0/856423786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05b7ba0 msgr2=0x55c7e05c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f590be00640 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f590be00640 1 -- 172.21.15.103:0/856423786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05bcc30 msgr2=0x55c7e05c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f590be00640 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05bcc30 0x55c7e05c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f590be00640 1 -- 172.21.15.103:0/856423786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7e05c7890 con 0x55c7e05b7260 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f5910a00640 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05bcc30 0x55c7e05c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f590be00640 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05c5d30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55c7e051c210 tx=0x7f58fc0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca7bb415d4ec6ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f58f9600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.270+0000 7f58f9600640 1 -- 172.21.15.103:0/856423786 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc039020 con 0x55c7e05b7260 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f591587a740 1 -- 172.21.15.103:0/856423786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58fc005520 con 0x55c7e05b7260 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f58f9600640 1 -- 172.21.15.103:0/856423786 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f58fc009e30 con 0x55c7e05b7260 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f58f9600640 1 -- 172.21.15.103:0/856423786 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f58fc044050 con 0x55c7e05b7260 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f591587a740 10 client.25270 fetch_fsmap learned FSMap version 24 2024-09-16T02:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f591587a740 1 -- 172.21.15.103:0/856423786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c7e05c8ba0 con 0x55c7e05b7260 2024-09-16T02:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f58f9600640 1 -- 172.21.15.103:0/856423786 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58fc005520 con 0x55c7e05b7260 2024-09-16T02:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f58f9600640 1 -- 172.21.15.103:0/856423786 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f58fc021040 con 0x55c7e05b7260 2024-09-16T02:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f591587a740 10 client.25270 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f591587a740 10 client.25270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f591587a740 20 client.25270 populate_metadata read hostname 'smithi103' 2024-09-16T02:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f591587a740 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c7e0637fd0 0x55c7e063a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f591587a740 4 client.25270 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T02:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.271+0000 7f591587a740 1 -- 172.21.15.103:0/856423786 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7e0523b80 con 0x55c7e0637fd0 2024-09-16T02:06:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.272+0000 7f5903e00640 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c7e0637fd0 0x55c7e063a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:06:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.272+0000 7f5903e00640 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c7e0637fd0 0x55c7e063a3f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7deed7e11f84951b server_cookie=892afbb8a540925 in_seq=0 out_seq=0 2024-09-16T02:06:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.272+0000 7f58f9600640 10 client.25270 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f58f9600640 1 -- 172.21.15.103:0/856423786 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7e0523b80 con 0x55c7e0637fd0 2024-09-16T02:06:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f58f9600640 10 client.25270 handle_command_reply: tid=0 2024-09-16T02:06:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 client.25270 shutdown 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 -- 172.21.15.103:0/856423786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 msgr2=0x55c7e05c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05c5d30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55c7e051c210 tx=0x7f58fc0056d0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 -- 172.21.15.103:0/856423786 shutdown_connections 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c7e0637fd0 0x55c7e063a3f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e05bcc30 0x55c7e05c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f58f9600640 0 client.25270 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c7e05b7ba0 0x55c7e05c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 --2- 172.21.15.103:0/856423786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7e05b7260 0x55c7e05c5d30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 -- 172.21.15.103:0/856423786 >> 172.21.15.103:0/856423786 conn(0x55c7e05b28d0 msgr2=0x55c7e05b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 -- 172.21.15.103:0/856423786 shutdown_connections 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.278+0000 7f591587a740 1 -- 172.21.15.103:0/856423786 wait complete. 2024-09-16T02:06:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f591587a740 20 client.25270 trim_cache size 0 max 16384 2024-09-16T02:06:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-16T02:06:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-16T02:06:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-16T02:06:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f5911400640 1 -- 172.21.15.103:0/3482859153 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f58f00a3350 msgr2=0x7f58f00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f5911400640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f58f00a3350 0x7f58f00a5810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58f4031440 tx=0x7f58f40315b0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f5911400640 1 -- 172.21.15.103:0/3482859153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590c109d10 msgr2=0x7f590c1a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f5911400640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590c109d10 0x7f590c1a15b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f590400ec50 tx=0x7f590400c6a0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f5911400640 1 -- 172.21.15.103:0/3482859153 shutdown_connections 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f5911400640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f58f00a3350 0x7f58f00a5810 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f5911400640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590c109d10 0x7f590c1a15b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f5911400640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f590c074f40 0x7f590c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f5911400640 1 --2- 172.21.15.103:0/3482859153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f590c074600 0x7f590c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.279+0000 7f5911400640 1 -- 172.21.15.103:0/3482859153 >> 172.21.15.103:0/3482859153 conn(0x7f590c0fd0d0 msgr2=0x7f590c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:06:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.280+0000 7f5911400640 1 -- 172.21.15.103:0/3482859153 shutdown_connections 2024-09-16T02:06:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:06:47.280+0000 7f5911400640 1 -- 172.21.15.103:0/3482859153 wait complete. 2024-09-16T02:06:47.998 INFO:tasks.workunit.client.0.smithi103.stderr:Updating files: 64% (8366/12889) Updating files: 65% (8378/12889) Updating files: 66% (8507/12889) Updating files: 67% (8636/12889) Updating files: 68% (8765/12889) Updating files: 69% (8894/12889) Updating files: 70% (9023/12889) Updating files: 71% (9152/12889) Updating files: 72% (9281/12889) Updating files: 73% (9409/12889) Updating files: 74% (9538/12889) Updating files: 75% (9667/12889) Updating files: 76% (9796/12889) Updating files: 77% (9925/12889) Updating files: 78% (10054/12889) Updating files: 79% (10183/12889) Updating files: 80% (10312/12889) Updating files: 81% (10441/12889) Updating files: 82% (10569/12889) Updating files: 83% (10698/12889) Updating files: 84% (10827/12889) Updating files: 85% (10956/12889) Updating files: 86% (11085/12889) Updating files: 87% (11214/12889) Updating files: 88% (11343/12889) Updating files: 89% (11472/12889) Updating files: 90% (11601/12889) Updating files: 91% (11729/12889) Updating files: 92% (11858/12889) Updating files: 93% (11987/12889) Updating files: 94% (12116/12889) Updating files: 95% (12245/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-16T02:06:48.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:47 smithi103 ceph-mon[30398]: pgmap v380: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:06:48.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:47 smithi136 ceph-mon[35323]: pgmap v380: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:06:48.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:47 smithi172 ceph-mon[35340]: pgmap v380: 97 pgs: 97 active+clean; 673 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:06:48.622 INFO:tasks.workunit.client.0.smithi103.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-16T02:06:48.622 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-16T02:06:48.622 INFO:tasks.workunit.client.0.smithi103.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr: git switch -c 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr:Or undo this operation with: 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr: git switch - 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T02:06:48.623 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-16T02:06:48.624 INFO:tasks.workunit.client.0.smithi103.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-16T02:06:48.636 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T02:06:48.721 INFO:tasks.workunit.client.0.smithi103.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T02:06:48.726 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T02:06:48.726 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T02:06:49.460 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T02:06:49.559 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T02:06:49.630 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T02:06:49.644 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T02:06:49.644 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T02:06:49.745 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T02:06:49.752 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T02:06:49.752 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T02:06:49.835 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-09-16T02:06:49.836 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-09-16T02:06:49.837 DEBUG:teuthology.orchestra.run.smithi103:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-09-16T02:06:49.928 INFO:tasks.workunit.client.0.smithi103.stderr:+ echo 'test large (16MB) dio write' 2024-09-16T02:06:49.928 INFO:tasks.workunit.client.0.smithi103.stdout:test large (16MB) dio write 2024-09-16T02:06:49.928 INFO:tasks.workunit.client.0.smithi103.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-09-16T02:06:50.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:49 smithi103 ceph-mon[30398]: pgmap v381: 97 pgs: 97 active+clean; 674 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-16T02:06:50.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:06:50.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:49 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:49 smithi136 ceph-mon[35323]: pgmap v381: 97 pgs: 97 active+clean; 674 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-16T02:06:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:06:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:49 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:49 smithi172 ceph-mon[35340]: pgmap v381: 97 pgs: 97 active+clean; 674 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-16T02:06:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:06:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:49 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:50.151 INFO:tasks.workunit.client.0.smithi103.stderr:1+0 records in 2024-09-16T02:06:50.151 INFO:tasks.workunit.client.0.smithi103.stderr:1+0 records out 2024-09-16T02:06:50.151 INFO:tasks.workunit.client.0.smithi103.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.219507 s, 76.4 MB/s 2024-09-16T02:06:50.153 INFO:tasks.workunit.client.0.smithi103.stdout:OK 2024-09-16T02:06:50.154 INFO:tasks.workunit.client.0.smithi103.stderr:+ echo OK 2024-09-16T02:06:50.154 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T02:06:50.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T02:06:50.221 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-09-16T02:06:50.221 DEBUG:teuthology.orchestra.run.smithi103:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-09-16T02:06:51.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:50 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:06:51.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:50 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:06:51.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:50 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:51.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:50 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:06:51.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:50 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:06:51.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:50 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:51.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:50 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:06:51.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:50 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:06:51.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:50 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:06:52.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:51 smithi103 ceph-mon[30398]: pgmap v382: 97 pgs: 97 active+clean; 674 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:06:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:51 smithi136 ceph-mon[35323]: pgmap v382: 97 pgs: 97 active+clean; 674 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:06:52.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:51 smithi172 ceph-mon[35340]: pgmap v382: 97 pgs: 97 active+clean; 674 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T02:06:54.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:53 smithi103 ceph-mon[30398]: pgmap v383: 97 pgs: 97 active+clean; 13 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.0 MiB/s wr, 6 op/s 2024-09-16T02:06:54.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:53 smithi136 ceph-mon[35323]: pgmap v383: 97 pgs: 97 active+clean; 13 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.0 MiB/s wr, 6 op/s 2024-09-16T02:06:54.044 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:53 smithi172 ceph-mon[35340]: pgmap v383: 97 pgs: 97 active+clean; 13 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.0 MiB/s wr, 6 op/s 2024-09-16T02:06:56.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:55 smithi103 ceph-mon[30398]: pgmap v384: 97 pgs: 97 active+clean; 13 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 551 op/s 2024-09-16T02:06:56.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:55 smithi136 ceph-mon[35323]: pgmap v384: 97 pgs: 97 active+clean; 13 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 551 op/s 2024-09-16T02:06:56.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:55 smithi172 ceph-mon[35340]: pgmap v384: 97 pgs: 97 active+clean; 13 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 551 op/s 2024-09-16T02:06:58.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:57 smithi103 ceph-mon[30398]: pgmap v385: 97 pgs: 97 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 550 op/s 2024-09-16T02:06:58.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:57 smithi136 ceph-mon[35323]: pgmap v385: 97 pgs: 97 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 550 op/s 2024-09-16T02:06:58.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:57 smithi172 ceph-mon[35340]: pgmap v385: 97 pgs: 97 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 550 op/s 2024-09-16T02:06:59.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:07:00.002 INFO:tasks.workunit.client.0.smithi103.stdout:.......... 2024-09-16T02:07:00.002 INFO:tasks.workunit.client.0.smithi103.stdout:do_o_direct_reads: SUCCESS 2024-09-16T02:07:00.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:59 smithi103 ceph-mon[30398]: pgmap v386: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-16T02:07:00.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:06:59 smithi103 ceph-mon[30398]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-16T02:07:00.006 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T02:07:00.007 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T02:07:00.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:59 smithi136 ceph-mon[35323]: pgmap v386: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-16T02:07:00.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:06:59 smithi136 ceph-mon[35323]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-16T02:07:00.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:59 smithi172 ceph-mon[35340]: pgmap v386: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-16T02:07:00.046 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:06:59 smithi172 ceph-mon[35340]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-16T02:07:00.120 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-09-16T02:07:00.120 DEBUG:teuthology.orchestra.run.smithi103:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-09-16T02:07:00.168 INFO:tasks.workunit.client.0.smithi103.stdout:test read from hole 2024-09-16T02:07:00.168 INFO:tasks.workunit.client.0.smithi103.stderr:+ echo 'test read from hole' 2024-09-16T02:07:00.168 INFO:tasks.workunit.client.0.smithi103.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-09-16T02:07:00.171 INFO:tasks.workunit.client.0.smithi103.stderr:0+0 records in 2024-09-16T02:07:00.171 INFO:tasks.workunit.client.0.smithi103.stderr:0+0 records out 2024-09-16T02:07:00.171 INFO:tasks.workunit.client.0.smithi103.stderr:0 bytes copied, 8.4701e-05 s, 0.0 kB/s 2024-09-16T02:07:00.172 INFO:tasks.workunit.client.0.smithi103.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-09-16T02:07:00.175 INFO:tasks.workunit.client.0.smithi103.stderr:2+0 records in 2024-09-16T02:07:00.175 INFO:tasks.workunit.client.0.smithi103.stderr:2+0 records out 2024-09-16T02:07:00.175 INFO:tasks.workunit.client.0.smithi103.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0019548 s, 524 kB/s 2024-09-16T02:07:00.176 INFO:tasks.workunit.client.0.smithi103.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-09-16T02:07:00.177 INFO:tasks.workunit.client.0.smithi103.stderr:2+0 records in 2024-09-16T02:07:00.177 INFO:tasks.workunit.client.0.smithi103.stderr:2+0 records out 2024-09-16T02:07:00.177 INFO:tasks.workunit.client.0.smithi103.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 9.094e-05 s, 11.3 MB/s 2024-09-16T02:07:00.178 INFO:tasks.workunit.client.0.smithi103.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-09-16T02:07:00.179 INFO:tasks.workunit.client.0.smithi103.stdout:other thing 2024-09-16T02:07:00.180 INFO:tasks.workunit.client.0.smithi103.stderr:+ echo 'other thing' 2024-09-16T02:07:00.180 INFO:tasks.workunit.client.0.smithi103.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-09-16T02:07:00.181 INFO:tasks.workunit.client.0.smithi103.stderr:1+0 records in 2024-09-16T02:07:00.181 INFO:tasks.workunit.client.0.smithi103.stderr:1+0 records out 2024-09-16T02:07:00.181 INFO:tasks.workunit.client.0.smithi103.stderr:500 bytes copied, 9.9078e-05 s, 5.0 MB/s 2024-09-16T02:07:00.182 INFO:tasks.workunit.client.0.smithi103.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-09-16T02:07:00.191 INFO:tasks.workunit.client.0.smithi103.stderr:0+1 records in 2024-09-16T02:07:00.191 INFO:tasks.workunit.client.0.smithi103.stderr:0+1 records out 2024-09-16T02:07:00.191 INFO:tasks.workunit.client.0.smithi103.stderr:500 bytes copied, 0.00649762 s, 77.0 kB/s 2024-09-16T02:07:00.191 INFO:tasks.workunit.client.0.smithi103.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-09-16T02:07:00.199 INFO:tasks.workunit.client.0.smithi103.stderr:0+1 records in 2024-09-16T02:07:00.199 INFO:tasks.workunit.client.0.smithi103.stderr:0+1 records out 2024-09-16T02:07:00.199 INFO:tasks.workunit.client.0.smithi103.stderr:500 bytes copied, 0.00564063 s, 88.6 kB/s 2024-09-16T02:07:00.199 INFO:tasks.workunit.client.0.smithi103.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-09-16T02:07:00.201 INFO:tasks.workunit.client.0.smithi103.stdout:OK 2024-09-16T02:07:00.202 INFO:tasks.workunit.client.0.smithi103.stderr:+ echo OK 2024-09-16T02:07:00.202 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T02:07:00.202 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T02:07:00.275 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-09-16T02:07:00.275 DEBUG:teuthology.orchestra.run.smithi103:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-09-16T02:07:00.344 INFO:tasks.workunit.client.0.smithi103.stdout:writing first 3 bytes of 10k file 2024-09-16T02:07:00.345 INFO:tasks.workunit.client.0.smithi103.stdout:reading O_DIRECT 2024-09-16T02:07:00.345 INFO:tasks.workunit.client.0.smithi103.stdout:got 10000 2024-09-16T02:07:00.345 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T02:07:00.345 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T02:07:00.381 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.410+0000 7fdad1800640 1 -- 172.21.15.103:0/2963102468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdacc101580 msgr2=0x7fdacc101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.410+0000 7fdad1800640 1 --2- 172.21.15.103:0/2963102468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdacc101580 0x7fdacc101a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009900 tx=0x7fdab402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.410+0000 7fdad1800640 1 -- 172.21.15.103:0/2963102468 shutdown_connections 2024-09-16T02:07:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.410+0000 7fdad1800640 1 --2- 172.21.15.103:0/2963102468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc101f40 0x7fdacc109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.410+0000 7fdad1800640 1 --2- 172.21.15.103:0/2963102468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdacc101580 0x7fdacc101a00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.410+0000 7fdad1800640 1 --2- 172.21.15.103:0/2963102468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdacc103050 0x7fdacc103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.410+0000 7fdad1800640 1 -- 172.21.15.103:0/2963102468 >> 172.21.15.103:0/2963102468 conn(0x7fdacc0fd0d0 msgr2=0x7fdacc0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.410+0000 7fdad1800640 1 -- 172.21.15.103:0/2963102468 shutdown_connections 2024-09-16T02:07:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.410+0000 7fdad1800640 1 -- 172.21.15.103:0/2963102468 wait complete. 2024-09-16T02:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.411+0000 7fdad1800640 1 Processor -- start 2024-09-16T02:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdad1800640 1 -- start start 2024-09-16T02:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdad1800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdacc101580 0x7fdacc197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdad1800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdacc101f40 0x7fdacc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdad1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc103050 0x7fdacc19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdad1800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc10d740 con 0x7fdacc101f40 2024-09-16T02:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdad1800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc10d5c0 con 0x7fdacc103050 2024-09-16T02:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdad1800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc10d8c0 con 0x7fdacc101580 2024-09-16T02:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdacc101f40 0x7fdacc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdacbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdacc101580 0x7fdacc197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdacbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdacc101580 0x7fdacc197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:51964/0 (socket says 172.21.15.103:51964) 2024-09-16T02:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdacbe00640 1 -- 172.21.15.103:0/1590857784 learned_addr learned my addr 172.21.15.103:0/1590857784 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.412+0000 7fdad0e00640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc103050 0x7fdacc19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdacbe00640 1 -- 172.21.15.103:0/1590857784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc103050 msgr2=0x7fdacc19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdacbe00640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc103050 0x7fdacc19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdacbe00640 1 -- 172.21.15.103:0/1590857784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdacc101f40 msgr2=0x7fdacc198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdacbe00640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdacc101f40 0x7fdacc198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdacbe00640 1 -- 172.21.15.103:0/1590857784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc19f960 con 0x7fdacc101580 2024-09-16T02:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdac3e00640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdacc101f40 0x7fdacc198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdacbe00640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdacc101580 0x7fdacc197d50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00c970 tx=0x7fdabc00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdac9600640 1 -- 172.21.15.103:0/1590857784 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc007c10 con 0x7fdacc101580 2024-09-16T02:07:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdac9600640 1 -- 172.21.15.103:0/1590857784 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdabc007db0 con 0x7fdacc101580 2024-09-16T02:07:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdac9600640 1 -- 172.21.15.103:0/1590857784 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00f940 con 0x7fdacc101580 2024-09-16T02:07:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.413+0000 7fdad1800640 1 -- 172.21.15.103:0/1590857784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdacc19fc50 con 0x7fdacc101580 2024-09-16T02:07:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.414+0000 7fdad1800640 1 -- 172.21.15.103:0/1590857784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdacc1a0150 con 0x7fdacc101580 2024-09-16T02:07:00.419 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-09-16T02:07:00.419 DEBUG:teuthology.orchestra.run.smithi103:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-09-16T02:07:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.417+0000 7fdad5bc2740 1 Processor -- start 2024-09-16T02:07:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.417+0000 7fdad5bc2740 1 -- start start 2024-09-16T02:07:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.417+0000 7fdac9600640 1 -- 172.21.15.103:0/1590857784 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdabc00fae0 con 0x7fdacc101580 2024-09-16T02:07:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.417+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 0x5612ea4e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.417+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.417+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e7040 0x5612ea4e9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.417+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5612ea442100 con 0x5612ea4e7040 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.417+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5612ea3ea1b0 con 0x5612ea4e1fb0 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.417+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5612ea43e0c0 con 0x5612ea4e1670 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdad0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 0x5612ea4e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdad0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 0x5612ea4e1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:51978/0 (socket says 172.21.15.103:51978) 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdad0e00640 1 -- 172.21.15.103:0/765596782 learned_addr learned my addr 172.21.15.103:0/765596782 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdac3e00640 1 --2- 172.21.15.103:0/765596782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e7040 0x5612ea4e9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdacbe00640 1 --2- 172.21.15.103:0/765596782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdad0e00640 1 -- 172.21.15.103:0/765596782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 msgr2=0x5612ea4e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdad0e00640 1 --2- 172.21.15.103:0/765596782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdad0e00640 1 -- 172.21.15.103:0/765596782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e7040 msgr2=0x5612ea4e9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdad0e00640 1 --2- 172.21.15.103:0/765596782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e7040 0x5612ea4e9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdad0e00640 1 -- 172.21.15.103:0/765596782 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612ea155210 con 0x5612ea4e1670 2024-09-16T02:07:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.418+0000 7fdad0e00640 1 --2- 172.21.15.103:0/765596782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 0x5612ea4e1a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdac4010370 tx=0x7fdac4010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b8fe4a299cb697 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.419+0000 7fdac1600640 1 -- 172.21.15.103:0/765596782 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4016070 con 0x5612ea4e1670 2024-09-16T02:07:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.419+0000 7fdac1600640 1 -- 172.21.15.103:0/765596782 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac4004d20 con 0x5612ea4e1670 2024-09-16T02:07:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.419+0000 7fdac1600640 1 -- 172.21.15.103:0/765596782 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4005480 con 0x5612ea4e1670 2024-09-16T02:07:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.419+0000 7fdac9600640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdab00a32c0 0x7fdab00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.419+0000 7fdac9600640 1 -- 172.21.15.103:0/1590857784 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fdabc0cfc50 con 0x7fdacc101580 2024-09-16T02:07:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.419+0000 7fdacbe00640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdab00a32c0 0x7fdab00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.420+0000 7fdad5bc2740 1 -- 172.21.15.103:0/765596782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 msgr2=0x5612ea4e1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.420+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/765596782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 0x5612ea4e1a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdac4010370 tx=0x7fdac4010830 comp rx=0 tx=0).stop 2024-09-16T02:07:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.420+0000 7fdad5bc2740 1 -- 172.21.15.103:0/765596782 shutdown_connections 2024-09-16T02:07:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.420+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/765596782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e7040 0x5612ea4e9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.420+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/765596782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.420+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/765596782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 0x5612ea4e1a70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.420+0000 7fdad5bc2740 1 -- 172.21.15.103:0/765596782 >> 172.21.15.103:0/765596782 conn(0x5612ea4d88d0 msgr2=0x5612ea4d8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.420+0000 7fdacbe00640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdab00a32c0 0x7fdab00a5780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0cda40 tx=0x7fdabc0d3730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.420+0000 7fdad5bc2740 1 -- 172.21.15.103:0/765596782 shutdown_connections 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.420+0000 7fdad5bc2740 1 -- 172.21.15.103:0/765596782 wait complete. 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.421+0000 7fdad5bc2740 1 -- start start 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.421+0000 7fdad5bc2740 1 Processor -- start 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.421+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4eb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.421+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.421+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e7040 0x5612ea4f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.421+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5612ea4e9fb0 con 0x5612ea4e1670 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.421+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5612ea3cb090 con 0x5612ea4e1fb0 2024-09-16T02:07:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.421+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5612ea4ea130 con 0x5612ea4e7040 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.421+0000 7fdad5bc2740 10 client.0 _ll_register_callbacks cb 0x5612ea41ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdad0e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4eb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdad0e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4eb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52664/0 (socket says 172.21.15.103:52664) 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdad0e00640 1 -- 172.21.15.103:0/1242399597 learned_addr learned my addr 172.21.15.103:0/1242399597 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdad0e00640 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e7040 0x5612ea4f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdac3e00640 1 -- 172.21.15.103:0/1242399597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e7040 msgr2=0x5612ea4f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdac3e00640 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e7040 0x5612ea4f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdac3e00640 1 -- 172.21.15.103:0/1242399597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 msgr2=0x5612ea4eb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdac3e00640 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4eb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdac3e00640 1 -- 172.21.15.103:0/1242399597 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ea221620 con 0x5612ea4e1fb0 2024-09-16T02:07:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdac3e00640 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4ebd20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdab402a910 tx=0x7fdab40053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e034ba83ce7aba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.422+0000 7fdaba000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:07:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.423+0000 7fdaba000640 1 -- 172.21.15.103:0/1242399597 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4035040 con 0x5612ea4e1fb0 2024-09-16T02:07:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.423+0000 7fdaba000640 1 -- 172.21.15.103:0/1242399597 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fdab4004970 con 0x5612ea4e1fb0 2024-09-16T02:07:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.423+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1242399597 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac4004d20 con 0x5612ea4e1fb0 2024-09-16T02:07:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.423+0000 7fdaba000640 1 -- 172.21.15.103:0/1242399597 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdab404d050 con 0x5612ea4e1fb0 2024-09-16T02:07:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.423+0000 7fdad5bc2740 10 client.25304 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.423+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1242399597 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5612ea4f5bc0 con 0x5612ea4e1fb0 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.423+0000 7fdaba000640 1 -- 172.21.15.103:0/1242399597 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac4004d20 con 0x5612ea4e1fb0 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.423+0000 7fdaba000640 1 -- 172.21.15.103:0/1242399597 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdab4044070 con 0x5612ea4e1fb0 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.424+0000 7fdad5bc2740 10 client.25304 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.424+0000 7fdad5bc2740 10 client.25304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.424+0000 7fdad5bc2740 20 client.25304 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.424+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5612ea4f7240 0x5612ea4f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.424+0000 7fdad5bc2740 4 client.25304 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.424+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1242399597 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612ea44f830 con 0x5612ea4f7240 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.424+0000 7fdad0e00640 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5612ea4f7240 0x5612ea4f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.424+0000 7fdad0e00640 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5612ea4f7240 0x5612ea4f9650 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb067440fd64662 server_cookie=9eea994c7c44e42a in_seq=0 out_seq=0 2024-09-16T02:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.424+0000 7fdaba000640 10 client.25304 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdaba000640 1 -- 172.21.15.103:0/1242399597 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612ea44f830 con 0x5612ea4f7240 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdaba000640 10 client.25304 handle_command_reply: tid=0 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdad5bc2740 1 client.25304 shutdown 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1242399597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 msgr2=0x5612ea4ebd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4ebd20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdab402a910 tx=0x7fdab40053b0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1242399597 shutdown_connections 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5612ea4f7240 0x5612ea4f9650 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e7040 0x5612ea4f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4ebd20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdaba000640 0 client.25304 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1242399597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4eb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.425+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1242399597 >> 172.21.15.103:0/1242399597 conn(0x5612ea4d88d0 msgr2=0x5612ea4da0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.426+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1242399597 shutdown_connections 2024-09-16T02:07:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.426+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1242399597 wait complete. 2024-09-16T02:07:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.426+0000 7fdad5bc2740 20 client.25304 trim_cache size 0 max 16384 2024-09-16T02:07:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.441+0000 7fdad5bc2740 1 Processor -- start 2024-09-16T02:07:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.441+0000 7fdad5bc2740 1 -- start start 2024-09-16T02:07:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4e6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e7040 0x5612ea4d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1a9670 con 0x5612ea4e1670 2024-09-16T02:07:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1a6f40 con 0x5612ea4e1fb0 2024-09-16T02:07:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1a4d90 con 0x5612ea4e7040 2024-09-16T02:07:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad0e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad0e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4e6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52674/0 (socket says 172.21.15.103:52674) 2024-09-16T02:07:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad0e00640 1 -- 172.21.15.103:0/542309429 learned_addr learned my addr 172.21.15.103:0/542309429 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdac3e00640 1 --2- 172.21.15.103:0/542309429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad0e00640 1 --2- 172.21.15.103:0/542309429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e7040 0x5612ea4d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad0e00640 1 -- 172.21.15.103:0/542309429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e7040 msgr2=0x5612ea4d9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad0e00640 1 --2- 172.21.15.103:0/542309429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e7040 0x5612ea4d9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad0e00640 1 -- 172.21.15.103:0/542309429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 msgr2=0x5612ea4f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.442+0000 7fdad0e00640 1 --2- 172.21.15.103:0/542309429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.443+0000 7fdad0e00640 1 -- 172.21.15.103:0/542309429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612ea4f9cd0 con 0x5612ea4e1670 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.443+0000 7fdac3e00640 1 --2- 172.21.15.103:0/542309429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.443+0000 7fdad0e00640 1 --2- 172.21.15.103:0/542309429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4e6880 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009c20 tx=0x7fdac4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa78fca650598c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.443+0000 7fdac0c00640 1 -- 172.21.15.103:0/542309429 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4030070 con 0x5612ea4e1670 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.443+0000 7fdac0c00640 1 -- 172.21.15.103:0/542309429 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac4004cc0 con 0x5612ea4e1670 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.443+0000 7fdac0c00640 1 -- 172.21.15.103:0/542309429 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4016070 con 0x5612ea4e1670 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.444+0000 7fdad5bc2740 1 -- 172.21.15.103:0/542309429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 msgr2=0x5612ea4e6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.444+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/542309429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4e6880 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009c20 tx=0x7fdac4009bb0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.444+0000 7fdad5bc2740 1 -- 172.21.15.103:0/542309429 shutdown_connections 2024-09-16T02:07:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.444+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/542309429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e7040 0x5612ea4d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.444+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/542309429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.444+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/542309429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1670 0x5612ea4e6880 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.444+0000 7fdad5bc2740 1 -- 172.21.15.103:0/542309429 >> 172.21.15.103:0/542309429 conn(0x5612ea4d88d0 msgr2=0x5612ea4d8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.444+0000 7fdad5bc2740 1 -- 172.21.15.103:0/542309429 shutdown_connections 2024-09-16T02:07:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.444+0000 7fdad5bc2740 1 -- 172.21.15.103:0/542309429 wait complete. 2024-09-16T02:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdad5bc2740 1 -- start start 2024-09-16T02:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdad5bc2740 1 Processor -- start 2024-09-16T02:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 0x5612ea4efc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdad5bc2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e7040 0x5612ea55ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1a5400 con 0x5612ea4e1fb0 2024-09-16T02:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1a6870 con 0x5612ea4e7040 2024-09-16T02:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdad5bc2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1a5ad0 con 0x5612ea4e1670 2024-09-16T02:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdad5bc2740 10 client.0 _ll_register_callbacks cb 0x7fdab40043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52676/0 (socket says 172.21.15.103:52676) 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdac3e00640 1 -- 172.21.15.103:0/1159023224 learned_addr learned my addr 172.21.15.103:0/1159023224 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.445+0000 7fdad0e00640 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 0x5612ea4efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdad0e00640 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e7040 0x5612ea55ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdac3e00640 1 -- 172.21.15.103:0/1159023224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 msgr2=0x5612ea4efc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdac3e00640 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 0x5612ea4efc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdac3e00640 1 -- 172.21.15.103:0/1159023224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e7040 msgr2=0x5612ea55ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdac3e00640 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e7040 0x5612ea55ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdac3e00640 1 -- 172.21.15.103:0/1159023224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ea4ea230 con 0x5612ea4e1fb0 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdad0e00640 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e7040 0x5612ea55ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdac3e00640 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f0160 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009900 tx=0x7fdab4003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ada60a52296148b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdaba000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdaba000640 1 -- 172.21.15.103:0/1159023224 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4035040 con 0x5612ea4e1fb0 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1159023224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac4004cc0 con 0x5612ea4e1fb0 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.446+0000 7fdaba000640 1 -- 172.21.15.103:0/1159023224 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fdab40316b0 con 0x5612ea4e1fb0 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdaba000640 1 -- 172.21.15.103:0/1159023224 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdab4000e60 con 0x5612ea4e1fb0 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdad5bc2740 10 client.15345 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1159023224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5612ea561590 con 0x5612ea4e1fb0 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdaba000640 1 -- 172.21.15.103:0/1159023224 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac4004cc0 con 0x5612ea4e1fb0 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdaba000640 1 -- 172.21.15.103:0/1159023224 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdab4044040 con 0x5612ea4e1fb0 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdad5bc2740 10 client.15345 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdad5bc2740 10 client.15345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdad5bc2740 20 client.15345 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5612ea561eb0 0x5612ea5642d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdad5bc2740 4 client.15345 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T02:07:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.447+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1159023224 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612ea449b80 con 0x5612ea561eb0 2024-09-16T02:07:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.448+0000 7fdad0e00640 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5612ea561eb0 0x5612ea5642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.448+0000 7fdad0e00640 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5612ea561eb0 0x5612ea5642d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd19438c96e17ceb server_cookie=93b465b20231de81 in_seq=0 out_seq=0 2024-09-16T02:07:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.448+0000 7fdaba000640 10 client.15345 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdaba000640 1 -- 172.21.15.103:0/1159023224 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612ea449b80 con 0x5612ea561eb0 2024-09-16T02:07:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdaba000640 10 client.15345 handle_command_reply: tid=0 2024-09-16T02:07:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 client.15345 shutdown 2024-09-16T02:07:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1159023224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1fb0 msgr2=0x5612ea4f0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f0160 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009900 tx=0x7fdab4003ed0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1159023224 shutdown_connections 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5612ea561eb0 0x5612ea5642d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5612ea4e7040 0x5612ea55ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612ea4e1fb0 0x5612ea4f0160 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 --2- 172.21.15.103:0/1159023224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612ea4e1670 0x5612ea4efc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdaba000640 0 client.15345 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1159023224 >> 172.21.15.103:0/1159023224 conn(0x5612ea4d88d0 msgr2=0x5612ea4d71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1159023224 shutdown_connections 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 1 -- 172.21.15.103:0/1159023224 wait complete. 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.467+0000 7fdad5bc2740 20 client.15345 trim_cache size 0 max 16384 2024-09-16T02:07:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-16T02:07:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-16T02:07:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-16T02:07:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-16T02:07:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 -- 172.21.15.103:0/1590857784 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdab00a32c0 msgr2=0x7fdab00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdab00a32c0 0x7fdab00a5780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0cda40 tx=0x7fdabc0d3730 comp rx=0 tx=0).stop 2024-09-16T02:07:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 -- 172.21.15.103:0/1590857784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdacc101580 msgr2=0x7fdacc197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdacc101580 0x7fdacc197d50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00c970 tx=0x7fdabc00ce30 comp rx=0 tx=0).stop 2024-09-16T02:07:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 -- 172.21.15.103:0/1590857784 shutdown_connections 2024-09-16T02:07:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fdab00a32c0 0x7fdab00a5780 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc103050 0x7fdacc19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdacc101f40 0x7fdacc198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 --2- 172.21.15.103:0/1590857784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdacc101580 0x7fdacc197d50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 -- 172.21.15.103:0/1590857784 >> 172.21.15.103:0/1590857784 conn(0x7fdacc0fd0d0 msgr2=0x7fdacc105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 -- 172.21.15.103:0/1590857784 shutdown_connections 2024-09-16T02:07:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:00.468+0000 7fdad1800640 1 -- 172.21.15.103:0/1590857784 wait complete. 2024-09-16T02:07:00.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.964+0000 7f0e7da00640 1 -- 172.21.15.103:0/646129953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e78074e30 msgr2=0x7f0e7810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.964+0000 7f0e7da00640 1 --2- 172.21.15.103:0/646129953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e78074e30 0x7f0e7810dd40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009860 tx=0x7f0e6402a3d0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.964+0000 7f0e7da00640 1 -- 172.21.15.103:0/646129953 shutdown_connections 2024-09-16T02:07:00.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.964+0000 7f0e7da00640 1 --2- 172.21.15.103:0/646129953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e7810e280 0x7f0e78110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.964+0000 7f0e7da00640 1 --2- 172.21.15.103:0/646129953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e78074e30 0x7f0e7810dd40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.964+0000 7f0e7da00640 1 --2- 172.21.15.103:0/646129953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e78074510 0x7f0e780748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.964+0000 7f0e7da00640 1 -- 172.21.15.103:0/646129953 >> 172.21.15.103:0/646129953 conn(0x7f0e780fd740 msgr2=0x7f0e780ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:00.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.964+0000 7f0e7da00640 1 -- 172.21.15.103:0/646129953 shutdown_connections 2024-09-16T02:07:00.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.964+0000 7f0e7da00640 1 -- 172.21.15.103:0/646129953 wait complete. 2024-09-16T02:07:00.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.965+0000 7f0e7da00640 1 Processor -- start 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.965+0000 7f0e7da00640 1 -- start start 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e7da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e78074510 0x7f0e7810dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e7da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e78074e30 0x7f0e7810ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e7da00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e7810e280 0x7f0e7810b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e7da00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78113f70 con 0x7f0e7810e280 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e7da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78113df0 con 0x7f0e78074e30 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e7da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781140f0 con 0x7f0e78074510 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e77e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e78074510 0x7f0e7810dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e77e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e78074510 0x7f0e7810dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:52042/0 (socket says 172.21.15.103:52042) 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e77e00640 1 -- 172.21.15.103:0/1681038923 learned_addr learned my addr 172.21.15.103:0/1681038923 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e7d000640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e7810e280 0x7f0e7810b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.966+0000 7f0e77400640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e78074e30 0x7f0e7810ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.967+0000 7f0e7d000640 1 -- 172.21.15.103:0/1681038923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e78074510 msgr2=0x7f0e7810dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.967+0000 7f0e7d000640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e78074510 0x7f0e7810dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.967+0000 7f0e7d000640 1 -- 172.21.15.103:0/1681038923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e78074e30 msgr2=0x7f0e7810ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.967+0000 7f0e7d000640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e78074e30 0x7f0e7810ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.967+0000 7f0e7d000640 1 -- 172.21.15.103:0/1681038923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e781aacb0 con 0x7f0e7810e280 2024-09-16T02:07:00.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.967+0000 7f0e77400640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e78074e30 0x7f0e7810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:00.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.967+0000 7f0e7d000640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e7810e280 0x7f0e7810b340 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0e70005fd0 tx=0x7f0e7000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:00.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.968+0000 7f0e74c00640 1 -- 172.21.15.103:0/1681038923 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000cdf0 con 0x7f0e7810e280 2024-09-16T02:07:00.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.968+0000 7f0e74c00640 1 -- 172.21.15.103:0/1681038923 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e70005020 con 0x7f0e7810e280 2024-09-16T02:07:00.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.968+0000 7f0e74c00640 1 -- 172.21.15.103:0/1681038923 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e700106d0 con 0x7f0e7810e280 2024-09-16T02:07:00.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.968+0000 7f0e7da00640 1 -- 172.21.15.103:0/1681038923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e781aae80 con 0x7f0e7810e280 2024-09-16T02:07:00.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.968+0000 7f0e7da00640 1 -- 172.21.15.103:0/1681038923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e781ab2e0 con 0x7f0e7810e280 2024-09-16T02:07:00.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.971+0000 7f0e74c00640 1 -- 172.21.15.103:0/1681038923 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0e700054d0 con 0x7f0e7810e280 2024-09-16T02:07:00.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.971+0000 7f0e81a49740 1 Processor -- start 2024-09-16T02:07:00.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e81a49740 1 -- start start 2024-09-16T02:07:00.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 0x560b2bbe8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 0x560b2bbf29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbf2f30 0x560b2bbf5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e81a49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b2bb4e3e0 con 0x560b2bbf2f30 2024-09-16T02:07:00.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e81a49740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b2baf5d80 con 0x560b2bbe8670 2024-09-16T02:07:00.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e81a49740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b2bb4a3a0 con 0x560b2bbe7d50 2024-09-16T02:07:00.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e77400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 0x560b2bbe8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e77400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 0x560b2bbe8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:52050/0 (socket says 172.21.15.103:52050) 2024-09-16T02:07:00.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e77400640 1 -- 172.21.15.103:0/467069962 learned_addr learned my addr 172.21.15.103:0/467069962 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:00.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e77e00640 1 --2- 172.21.15.103:0/467069962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 0x560b2bbf29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.972+0000 7f0e77400640 1 --2- 172.21.15.103:0/467069962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbf2f30 0x560b2bbf5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e77e00640 1 -- 172.21.15.103:0/467069962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 msgr2=0x560b2bbe8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e77e00640 1 --2- 172.21.15.103:0/467069962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 0x560b2bbe8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e77e00640 1 -- 172.21.15.103:0/467069962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbf2f30 msgr2=0x560b2bbf5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e77e00640 1 --2- 172.21.15.103:0/467069962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbf2f30 0x560b2bbf5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e77e00640 1 -- 172.21.15.103:0/467069962 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2bb14ae0 con 0x560b2bbe8670 2024-09-16T02:07:00.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e74c00640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0e5c0a3240 0x7f0e5c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e74c00640 1 -- 172.21.15.103:0/1681038923 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f0e700cffd0 con 0x7f0e7810e280 2024-09-16T02:07:00.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e77e00640 1 --2- 172.21.15.103:0/467069962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 0x560b2bbf29f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009ca0 tx=0x7f0e68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e04b8ab97e5faf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:00.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e7d000640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0e5c0a3240 0x7f0e5c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e6d600640 1 -- 172.21.15.103:0/467069962 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68004230 con 0x560b2bbe8670 2024-09-16T02:07:00.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e6d600640 1 -- 172.21.15.103:0/467069962 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e680043d0 con 0x560b2bbe8670 2024-09-16T02:07:00.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.973+0000 7f0e6d600640 1 -- 172.21.15.103:0/467069962 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e680197c0 con 0x560b2bbe8670 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.974+0000 7f0e81a49740 1 -- 172.21.15.103:0/467069962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 msgr2=0x560b2bbf29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.974+0000 7f0e7d000640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0e5c0a3240 0x7f0e5c0a5700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0e700a4cb0 tx=0x7f0e7000cf90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.974+0000 7f0e81a49740 1 --2- 172.21.15.103:0/467069962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 0x560b2bbf29f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009ca0 tx=0x7f0e68011810 comp rx=0 tx=0).stop 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.974+0000 7f0e81a49740 1 -- 172.21.15.103:0/467069962 shutdown_connections 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.974+0000 7f0e81a49740 1 --2- 172.21.15.103:0/467069962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbf2f30 0x560b2bbf5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.974+0000 7f0e81a49740 1 --2- 172.21.15.103:0/467069962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 0x560b2bbf29f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.974+0000 7f0e81a49740 1 --2- 172.21.15.103:0/467069962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 0x560b2bbe8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.974+0000 7f0e81a49740 1 -- 172.21.15.103:0/467069962 >> 172.21.15.103:0/467069962 conn(0x560b2bbe42e0 msgr2=0x560b2bbe46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.975+0000 7f0e81a49740 1 -- 172.21.15.103:0/467069962 shutdown_connections 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.975+0000 7f0e81a49740 1 -- 172.21.15.103:0/467069962 wait complete. 2024-09-16T02:07:00.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.975+0000 7f0e81a49740 1 -- start start 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.975+0000 7f0e81a49740 1 Processor -- start 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbefdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe8670 0x560b2bbf0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 0x560b2bbf0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e81a49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b2bbf5ba0 con 0x560b2bbe7d50 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e81a49740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b2bb090f0 con 0x560b2bbf2f30 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e81a49740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b2bbf5d20 con 0x560b2bbe8670 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e81a49740 10 client.0 _ll_register_callbacks cb 0x560b2bafa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e77400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbefdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e77e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe8670 0x560b2bbf0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e77400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbefdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52732/0 (socket says 172.21.15.103:52732) 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e77400640 1 -- 172.21.15.103:0/792478599 learned_addr learned my addr 172.21.15.103:0/792478599 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:00.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.976+0000 7f0e77400640 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 0x560b2bbf0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.977+0000 7f0e77e00640 1 -- 172.21.15.103:0/792478599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 msgr2=0x560b2bbf0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.977+0000 7f0e77e00640 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 0x560b2bbf0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.977+0000 7f0e77e00640 1 -- 172.21.15.103:0/792478599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 msgr2=0x560b2bbefdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:00.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.977+0000 7f0e77e00640 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbefdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.977+0000 7f0e77e00640 1 -- 172.21.15.103:0/792478599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2b8bf3c0 con 0x560b2bbe8670 2024-09-16T02:07:00.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.977+0000 7f0e77e00640 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe8670 0x560b2bbf0300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0e68000c00 tx=0x7f0e68009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e71fe7bc22d1ef8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:00.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.977+0000 7f0e62000640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:07:00.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.977+0000 7f0e62000640 1 -- 172.21.15.103:0/792478599 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68028020 con 0x560b2bbe8670 2024-09-16T02:07:00.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.977+0000 7f0e81a49740 1 -- 172.21.15.103:0/792478599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e680043d0 con 0x560b2bbe8670 2024-09-16T02:07:00.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.977+0000 7f0e62000640 1 -- 172.21.15.103:0/792478599 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f0e68003870 con 0x560b2bbe8670 2024-09-16T02:07:00.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e62000640 1 -- 172.21.15.103:0/792478599 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e68033050 con 0x560b2bbe8670 2024-09-16T02:07:00.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e81a49740 10 client.25300 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e81a49740 1 -- 172.21.15.103:0/792478599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560b2bc017e0 con 0x560b2bbe8670 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e62000640 1 -- 172.21.15.103:0/792478599 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e680043d0 con 0x560b2bbe8670 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e62000640 1 -- 172.21.15.103:0/792478599 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0e68018070 con 0x560b2bbe8670 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e81a49740 10 client.25300 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e81a49740 10 client.25300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e81a49740 20 client.25300 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e81a49740 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x560b2bc02bf0 0x560b2bc04fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e81a49740 4 client.25300 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.978+0000 7f0e81a49740 1 -- 172.21.15.103:0/792478599 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b2bb5b950 con 0x560b2bc02bf0 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.979+0000 7f0e77400640 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x560b2bc02bf0 0x560b2bc04fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.979+0000 7f0e77400640 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x560b2bc02bf0 0x560b2bc04fe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd3f0bb93773e60 server_cookie=9c99e87c06771405 in_seq=0 out_seq=0 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.979+0000 7f0e62000640 10 client.25300 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:00.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e62000640 1 -- 172.21.15.103:0/792478599 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b2bb5b950 con 0x560b2bc02bf0 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e62000640 10 client.25300 handle_command_reply: tid=0 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 client.25300 shutdown 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 -- 172.21.15.103:0/792478599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe8670 msgr2=0x560b2bbf0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe8670 0x560b2bbf0300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0e68000c00 tx=0x7f0e68009f90 comp rx=0 tx=0).stop 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 -- 172.21.15.103:0/792478599 shutdown_connections 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x560b2bc02bf0 0x560b2bc04fe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 0x560b2bbf0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe8670 0x560b2bbf0300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 --2- 172.21.15.103:0/792478599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbefdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 -- 172.21.15.103:0/792478599 >> 172.21.15.103:0/792478599 conn(0x560b2bbe42e0 msgr2=0x560b2bbe9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e62000640 0 client.25300 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 -- 172.21.15.103:0/792478599 shutdown_connections 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.980+0000 7f0e81a49740 1 -- 172.21.15.103:0/792478599 wait complete. 2024-09-16T02:07:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.981+0000 7f0e81a49740 20 client.25300 trim_cache size 0 max 16384 2024-09-16T02:07:01.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.998+0000 7f0e81a49740 1 Processor -- start 2024-09-16T02:07:01.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.998+0000 7f0e81a49740 1 -- start start 2024-09-16T02:07:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 0x560b2bbee6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe8670 0x560b2bbf0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 0x560b2bc0a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e81a49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b6490 con 0x560b2bbe8670 2024-09-16T02:07:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e81a49740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b3d60 con 0x560b2bbf2f30 2024-09-16T02:07:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e81a49740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b1bb0 con 0x560b2bbe7d50 2024-09-16T02:07:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe8670 0x560b2bbf0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe8670 0x560b2bbf0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52738/0 (socket says 172.21.15.103:52738) 2024-09-16T02:07:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77e00640 1 -- 172.21.15.103:0/1786624307 learned_addr learned my addr 172.21.15.103:0/1786624307 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77400640 1 --2- 172.21.15.103:0/1786624307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 0x560b2bbee6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77400640 1 --2- 172.21.15.103:0/1786624307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 0x560b2bc0a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77e00640 1 -- 172.21.15.103:0/1786624307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 msgr2=0x560b2bbee6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77e00640 1 --2- 172.21.15.103:0/1786624307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 0x560b2bbee6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77e00640 1 -- 172.21.15.103:0/1786624307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 msgr2=0x560b2bc0a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77e00640 1 --2- 172.21.15.103:0/1786624307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 0x560b2bc0a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77e00640 1 -- 172.21.15.103:0/1786624307 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2bbe4110 con 0x560b2bbe8670 2024-09-16T02:07:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:00.999+0000 7f0e77400640 1 --2- 172.21.15.103:0/1786624307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 0x560b2bc0a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.000+0000 7f0e77e00640 1 --2- 172.21.15.103:0/1786624307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe8670 0x560b2bbf0490 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0e68004c30 tx=0x7f0e68001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76c3162828f0b05 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.000+0000 7f0e6cc00640 1 -- 172.21.15.103:0/1786624307 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68028040 con 0x560b2bbe8670 2024-09-16T02:07:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.000+0000 7f0e6cc00640 1 -- 172.21.15.103:0/1786624307 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e680043a0 con 0x560b2bbe8670 2024-09-16T02:07:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.000+0000 7f0e6cc00640 1 -- 172.21.15.103:0/1786624307 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6801f040 con 0x560b2bbe8670 2024-09-16T02:07:01.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 -- 172.21.15.103:0/1786624307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe8670 msgr2=0x560b2bbf0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:01.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 --2- 172.21.15.103:0/1786624307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe8670 0x560b2bbf0490 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0e68004c30 tx=0x7f0e68001e50 comp rx=0 tx=0).stop 2024-09-16T02:07:01.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 -- 172.21.15.103:0/1786624307 shutdown_connections 2024-09-16T02:07:01.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 --2- 172.21.15.103:0/1786624307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbf2f30 0x560b2bc0a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 --2- 172.21.15.103:0/1786624307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe8670 0x560b2bbf0490 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 --2- 172.21.15.103:0/1786624307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbe7d50 0x560b2bbee6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 -- 172.21.15.103:0/1786624307 >> 172.21.15.103:0/1786624307 conn(0x560b2bbe42e0 msgr2=0x560b2bbefd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 -- 172.21.15.103:0/1786624307 shutdown_connections 2024-09-16T02:07:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 -- 172.21.15.103:0/1786624307 wait complete. 2024-09-16T02:07:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 -- start start 2024-09-16T02:07:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.001+0000 7f0e81a49740 1 Processor -- start 2024-09-16T02:07:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbfaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 0x560b2bbfb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e81a49740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbf2f30 0x560b2bbf7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e81a49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b2220 con 0x560b2bbe7d50 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e81a49740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b3690 con 0x560b2bbe8670 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e81a49740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b28f0 con 0x560b2bbf2f30 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e81a49740 10 client.0 _ll_register_callbacks cb 0x7f0e68009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e77400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbfaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e77400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbfaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52752/0 (socket says 172.21.15.103:52752) 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e77e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 0x560b2bbfb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e77400640 1 -- 172.21.15.103:0/2956242403 learned_addr learned my addr 172.21.15.103:0/2956242403 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.002+0000 7f0e77400640 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbf2f30 0x560b2bbf7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e77e00640 1 -- 172.21.15.103:0/2956242403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbf2f30 msgr2=0x560b2bbf7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e77e00640 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbf2f30 0x560b2bbf7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e77e00640 1 -- 172.21.15.103:0/2956242403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 msgr2=0x560b2bbfaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e77e00640 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbfaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e77e00640 1 -- 172.21.15.103:0/2956242403 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2bbe8be0 con 0x560b2bbe8670 2024-09-16T02:07:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e77400640 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbfaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e77e00640 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 0x560b2bbfb4b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0e68011850 tx=0x7f0e680059e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=298499625ad26bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e62000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:07:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e62000640 1 -- 172.21.15.103:0/2956242403 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68028040 con 0x560b2bbe8670 2024-09-16T02:07:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e81a49740 1 -- 172.21.15.103:0/2956242403 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e680043a0 con 0x560b2bbe8670 2024-09-16T02:07:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.003+0000 7f0e62000640 1 -- 172.21.15.103:0/2956242403 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f0e68004850 con 0x560b2bbe8670 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e62000640 1 -- 172.21.15.103:0/2956242403 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e68045020 con 0x560b2bbe8670 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e81a49740 10 client.25328 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e81a49740 1 -- 172.21.15.103:0/2956242403 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560b2bc6dc80 con 0x560b2bbe8670 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e62000640 1 -- 172.21.15.103:0/2956242403 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e680043a0 con 0x560b2bbe8670 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e62000640 1 -- 172.21.15.103:0/2956242403 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0e68018070 con 0x560b2bbe8670 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e81a49740 10 client.25328 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e81a49740 10 client.25328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e81a49740 20 client.25328 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e81a49740 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x560b2bc6e1e0 0x560b2bc70600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e81a49740 4 client.25328 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.004+0000 7f0e81a49740 1 -- 172.21.15.103:0/2956242403 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b2bb55e60 con 0x560b2bc6e1e0 2024-09-16T02:07:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e77400640 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x560b2bc6e1e0 0x560b2bc70600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e77400640 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x560b2bc6e1e0 0x560b2bc70600 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918ed1c428a24f81 server_cookie=e02463efe9ac29c7 in_seq=0 out_seq=0 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e62000640 10 client.25328 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e62000640 1 -- 172.21.15.103:0/2956242403 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560b2bb55e60 con 0x560b2bc6e1e0 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e62000640 10 client.25328 handle_command_reply: tid=0 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e81a49740 1 client.25328 shutdown 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e81a49740 1 -- 172.21.15.103:0/2956242403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 msgr2=0x560b2bbfb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e81a49740 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 0x560b2bbfb4b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0e68011850 tx=0x7f0e680059e0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e81a49740 1 -- 172.21.15.103:0/2956242403 shutdown_connections 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e81a49740 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x560b2bc6e1e0 0x560b2bc70600 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e81a49740 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b2bbf2f30 0x560b2bbf7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e81a49740 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b2bbe8670 0x560b2bbfb4b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e81a49740 1 --2- 172.21.15.103:0/2956242403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b2bbe7d50 0x560b2bbfaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.005+0000 7f0e81a49740 1 -- 172.21.15.103:0/2956242403 >> 172.21.15.103:0/2956242403 conn(0x560b2bbe42e0 msgr2=0x560b2bbe9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.006+0000 7f0e62000640 0 client.25328 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.006+0000 7f0e81a49740 1 -- 172.21.15.103:0/2956242403 shutdown_connections 2024-09-16T02:07:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.006+0000 7f0e81a49740 1 -- 172.21.15.103:0/2956242403 wait complete. 2024-09-16T02:07:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.006+0000 7f0e81a49740 20 client.25328 trim_cache size 0 max 16384 2024-09-16T02:07:01.014 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:07:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-16T02:07:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-16T02:07:01.014 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:07:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.006+0000 7f0e7da00640 1 -- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0e5c0a3240 msgr2=0x7f0e5c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.006+0000 7f0e7da00640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0e5c0a3240 0x7f0e5c0a5700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0e700a4cb0 tx=0x7f0e7000cf90 comp rx=0 tx=0).stop 2024-09-16T02:07:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.006+0000 7f0e7da00640 1 -- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e7810e280 msgr2=0x7f0e7810b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.006+0000 7f0e7da00640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e7810e280 0x7f0e7810b340 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0e70005fd0 tx=0x7f0e7000ef10 comp rx=0 tx=0).stop 2024-09-16T02:07:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.007+0000 7f0e7da00640 1 -- 172.21.15.103:0/1681038923 shutdown_connections 2024-09-16T02:07:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.007+0000 7f0e7da00640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f0e5c0a3240 0x7f0e5c0a5700 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.007+0000 7f0e7da00640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e7810e280 0x7f0e7810b340 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.007+0000 7f0e7da00640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e78074e30 0x7f0e7810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.007+0000 7f0e7da00640 1 --2- 172.21.15.103:0/1681038923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e78074510 0x7f0e7810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.007+0000 7f0e7da00640 1 -- 172.21.15.103:0/1681038923 >> 172.21.15.103:0/1681038923 conn(0x7f0e780fd740 msgr2=0x7f0e780fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.007+0000 7f0e7da00640 1 -- 172.21.15.103:0/1681038923 shutdown_connections 2024-09-16T02:07:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:01.007+0000 7f0e7da00640 1 -- 172.21.15.103:0/1681038923 wait complete. 2024-09-16T02:07:01.895 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T02:07:01.896 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:07:01.985 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:01 smithi103 ceph-mon[30398]: pgmap v387: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-16T02:07:02.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:01 smithi136 ceph-mon[35323]: pgmap v387: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-16T02:07:02.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:01 smithi172 ceph-mon[35340]: pgmap v387: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-16T02:07:02.317 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:02.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.884+0000 7f9076a00640 1 -- 172.21.15.103:0/3191963612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070074e30 msgr2=0x7f907010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:02.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.884+0000 7f9076a00640 1 --2- 172.21.15.103:0/3191963612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070074e30 0x7f907010dd40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f905c009900 tx=0x7f905c02a480 comp rx=0 tx=0).stop 2024-09-16T02:07:02.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.884+0000 7f9076a00640 1 -- 172.21.15.103:0/3191963612 shutdown_connections 2024-09-16T02:07:02.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.884+0000 7f9076a00640 1 --2- 172.21.15.103:0/3191963612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f907010e280 0x7f9070110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:02.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.884+0000 7f9076a00640 1 --2- 172.21.15.103:0/3191963612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070074e30 0x7f907010dd40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:02.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.884+0000 7f9076a00640 1 --2- 172.21.15.103:0/3191963612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070074510 0x7f90700748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:02.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.884+0000 7f9076a00640 1 -- 172.21.15.103:0/3191963612 >> 172.21.15.103:0/3191963612 conn(0x7f90700fd740 msgr2=0x7f90700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:02.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.884+0000 7f9076a00640 1 -- 172.21.15.103:0/3191963612 shutdown_connections 2024-09-16T02:07:02.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.884+0000 7f9076a00640 1 -- 172.21.15.103:0/3191963612 wait complete. 2024-09-16T02:07:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.885+0000 7f9076a00640 1 Processor -- start 2024-09-16T02:07:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.885+0000 7f9076a00640 1 -- start start 2024-09-16T02:07:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9076a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9070074510 0x7f907019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9076a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070074e30 0x7f907019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9076a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f907010e280 0x7f907019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9076a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9070113fb0 con 0x7f907010e280 2024-09-16T02:07:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9076a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9070113e30 con 0x7f9070074510 2024-09-16T02:07:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9076a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9070114130 con 0x7f9070074e30 2024-09-16T02:07:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9076000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f907010e280 0x7f907019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9074c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070074e30 0x7f907019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9075600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9070074510 0x7f907019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9076000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f907010e280 0x7f907019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52774/0 (socket says 172.21.15.103:52774) 2024-09-16T02:07:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9076000640 1 -- 172.21.15.103:0/361686679 learned_addr learned my addr 172.21.15.103:0/361686679 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.886+0000 7f9074c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070074e30 0x7f907019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:52090/0 (socket says 172.21.15.103:52090) 2024-09-16T02:07:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.887+0000 7f9076000640 1 -- 172.21.15.103:0/361686679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070074e30 msgr2=0x7f907019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.887+0000 7f9076000640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070074e30 0x7f907019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.887+0000 7f9076000640 1 -- 172.21.15.103:0/361686679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9070074510 msgr2=0x7f907019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.887+0000 7f9076000640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9070074510 0x7f907019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.887+0000 7f9076000640 1 -- 172.21.15.103:0/361686679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907019f4f0 con 0x7f907010e280 2024-09-16T02:07:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.887+0000 7f9076000640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f907010e280 0x7f907019ee10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f906800eb40 tx=0x7f906800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.887+0000 7f906e000640 1 -- 172.21.15.103:0/361686679 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068004b20 con 0x7f907010e280 2024-09-16T02:07:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.887+0000 7f906e000640 1 -- 172.21.15.103:0/361686679 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9068004cc0 con 0x7f907010e280 2024-09-16T02:07:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.887+0000 7f9076a00640 1 -- 172.21.15.103:0/361686679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907019f7e0 con 0x7f907010e280 2024-09-16T02:07:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.887+0000 7f906e000640 1 -- 172.21.15.103:0/361686679 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068010850 con 0x7f907010e280 2024-09-16T02:07:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.888+0000 7f9076a00640 1 -- 172.21.15.103:0/361686679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a70c0 con 0x7f907010e280 2024-09-16T02:07:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.889+0000 7f9076a00640 1 -- 172.21.15.103:0/361686679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90700748f0 con 0x7f907010e280 2024-09-16T02:07:02.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.891+0000 7f906e000640 1 -- 172.21.15.103:0/361686679 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9068005020 con 0x7f907010e280 2024-09-16T02:07:02.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.892+0000 7f906e000640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90580a3240 0x7f90580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:02.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.892+0000 7f906e000640 1 -- 172.21.15.103:0/361686679 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f90680d0110 con 0x7f907010e280 2024-09-16T02:07:02.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.892+0000 7f9075600640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90580a3240 0x7f90580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:02.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.893+0000 7f9075600640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90580a3240 0x7f90580a5700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9064009bd0 tx=0x7f9064005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:02.896+0000 7f906e000640 1 -- 172.21.15.103:0/361686679 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f906809cdb0 con 0x7f907010e280 2024-09-16T02:07:03.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.112+0000 7f9076a00640 1 -- 172.21.15.103:0/361686679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9070109460 con 0x7f907010e280 2024-09-16T02:07:03.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.113+0000 7f906e000640 1 -- 172.21.15.103:0/361686679 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7f906809c6b0 con 0x7f907010e280 2024-09-16T02:07:03.116 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:07:03.116 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":24,"btime":"2024-09-16T02:05:32:550231+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:05:31.661572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24506,24505]},"id":1}]} 2024-09-16T02:07:03.116 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 24 2024-09-16T02:07:03.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.116+0000 7f9076a00640 1 -- 172.21.15.103:0/361686679 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90580a3240 msgr2=0x7f90580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:03.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.116+0000 7f9076a00640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90580a3240 0x7f90580a5700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9064009bd0 tx=0x7f9064005f90 comp rx=0 tx=0).stop 2024-09-16T02:07:03.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.116+0000 7f9076a00640 1 -- 172.21.15.103:0/361686679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f907010e280 msgr2=0x7f907019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:03.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.116+0000 7f9076a00640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f907010e280 0x7f907019ee10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f906800eb40 tx=0x7f906800c2d0 comp rx=0 tx=0).stop 2024-09-16T02:07:03.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.117+0000 7f9076a00640 1 -- 172.21.15.103:0/361686679 shutdown_connections 2024-09-16T02:07:03.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.117+0000 7f9076a00640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f90580a3240 0x7f90580a5700 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:03.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.117+0000 7f9076a00640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f907010e280 0x7f907019ee10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:03.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.117+0000 7f9076a00640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070074e30 0x7f907019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:03.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.117+0000 7f9076a00640 1 --2- 172.21.15.103:0/361686679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9070074510 0x7f907019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:03.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.117+0000 7f9076a00640 1 -- 172.21.15.103:0/361686679 >> 172.21.15.103:0/361686679 conn(0x7f90700fd740 msgr2=0x7f90700fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:03.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.117+0000 7f9076a00640 1 -- 172.21.15.103:0/361686679 shutdown_connections 2024-09-16T02:07:03.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:03.117+0000 7f9076a00640 1 -- 172.21.15.103:0/361686679 wait complete. 2024-09-16T02:07:03.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:03 smithi103 ceph-mon[30398]: pgmap v388: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-16T02:07:03.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:03 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/361686679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:07:03.885 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T02:07:04.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:03 smithi136 ceph-mon[35323]: pgmap v388: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-16T02:07:04.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:03 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/361686679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:07:04.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:03 smithi172 ceph-mon[35340]: pgmap v388: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.4 MiB/s wr, 1.68k op/s 2024-09-16T02:07:04.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:03 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/361686679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:07:04.286 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:04.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.839+0000 7fcae6600640 1 -- 172.21.15.103:0/1663753082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae0109140 msgr2=0x7fcae0110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:04.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.839+0000 7fcae6600640 1 --2- 172.21.15.103:0/1663753082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae0109140 0x7fcae0110cc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009ca0 tx=0x7fcad802a8c0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.839+0000 7fcae6600640 1 -- 172.21.15.103:0/1663753082 shutdown_connections 2024-09-16T02:07:04.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.839+0000 7fcae6600640 1 --2- 172.21.15.103:0/1663753082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae0109140 0x7fcae0110cc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.839+0000 7fcae6600640 1 --2- 172.21.15.103:0/1663753082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae01049c0 0x7fcae0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.839+0000 7fcae6600640 1 --2- 172.21.15.103:0/1663753082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae0104010 0x7fcae01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.839+0000 7fcae6600640 1 -- 172.21.15.103:0/1663753082 >> 172.21.15.103:0/1663753082 conn(0x7fcae00ffa50 msgr2=0x7fcae0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:04.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.839+0000 7fcae6600640 1 -- 172.21.15.103:0/1663753082 shutdown_connections 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.839+0000 7fcae6600640 1 -- 172.21.15.103:0/1663753082 wait complete. 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.840+0000 7fcae6600640 1 Processor -- start 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.840+0000 7fcae6600640 1 -- start start 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.840+0000 7fcae6600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae0104010 0x7fcae019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.840+0000 7fcae6600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae01049c0 0x7fcae019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.840+0000 7fcae6600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae0109140 0x7fcae019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.840+0000 7fcae6600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcae0114150 con 0x7fcae0109140 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.840+0000 7fcae6600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcae0113fd0 con 0x7fcae01049c0 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.840+0000 7fcae6600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01142d0 con 0x7fcae0104010 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae0109140 0x7fcae019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae01049c0 0x7fcae019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae01049c0 0x7fcae019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:47616/0 (socket says 172.21.15.103:47616) 2024-09-16T02:07:04.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadea00640 1 -- 172.21.15.103:0/1717048309 learned_addr learned my addr 172.21.15.103:0/1717048309 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:04.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadf400640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae0104010 0x7fcae019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadea00640 1 -- 172.21.15.103:0/1717048309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae0104010 msgr2=0x7fcae019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:04.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadea00640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae0104010 0x7fcae019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadea00640 1 -- 172.21.15.103:0/1717048309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae0109140 msgr2=0x7fcae019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:04.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadea00640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae0109140 0x7fcae019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadea00640 1 -- 172.21.15.103:0/1717048309 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae019f640 con 0x7fcae01049c0 2024-09-16T02:07:04.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.841+0000 7fcadfe00640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae0109140 0x7fcae019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:04.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.842+0000 7fcadea00640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae01049c0 0x7fcae019abd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00e9f0 tx=0x7fcacc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:04.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.842+0000 7fcad7e00640 1 -- 172.21.15.103:0/1717048309 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00cd80 con 0x7fcae01049c0 2024-09-16T02:07:04.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.842+0000 7fcad7e00640 1 -- 172.21.15.103:0/1717048309 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcacc010910 con 0x7fcae01049c0 2024-09-16T02:07:04.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.842+0000 7fcae6600640 1 -- 172.21.15.103:0/1717048309 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae019f930 con 0x7fcae01049c0 2024-09-16T02:07:04.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.842+0000 7fcad7e00640 1 -- 172.21.15.103:0/1717048309 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc018990 con 0x7fcae01049c0 2024-09-16T02:07:04.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.842+0000 7fcae6600640 1 -- 172.21.15.103:0/1717048309 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae01a7210 con 0x7fcae01049c0 2024-09-16T02:07:04.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.845+0000 7fcae9073740 1 Processor -- start 2024-09-16T02:07:04.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.845+0000 7fcae9073740 1 -- start start 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.845+0000 7fcae9073740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 0x561cb28c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.845+0000 7fcae9073740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.845+0000 7fcae9073740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28cbe50 0x561cb28ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcae9073740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561cb28273e0 con 0x561cb28c0c70 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcae9073740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cb27ced80 con 0x561cb28c1590 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcae9073740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561cb28233a0 con 0x561cb28cbe50 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcadfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 0x561cb28c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcadf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcadf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:47620/0 (socket says 172.21.15.103:47620) 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcadf400640 1 -- 172.21.15.103:0/2794119076 learned_addr learned my addr 172.21.15.103:0/2794119076 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcadf400640 1 -- 172.21.15.103:0/2794119076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28cbe50 msgr2=0x561cb28ce240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcadf400640 1 --2- 172.21.15.103:0/2794119076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28cbe50 0x561cb28ce240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcadf400640 1 -- 172.21.15.103:0/2794119076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 msgr2=0x561cb28c1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcadf400640 1 --2- 172.21.15.103:0/2794119076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 0x561cb28c1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.846+0000 7fcadf400640 1 -- 172.21.15.103:0/2794119076 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb27edae0 con 0x561cb28c1590 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.847+0000 7fcadf400640 1 --2- 172.21.15.103:0/2794119076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28cb910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009cf0 tx=0x7fcad0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b7b70007c4ce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.847+0000 7fcad4c00640 1 -- 172.21.15.103:0/2794119076 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0004230 con 0x561cb28c1590 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.847+0000 7fcad4c00640 1 -- 172.21.15.103:0/2794119076 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad00043d0 con 0x561cb28c1590 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.847+0000 7fcad4c00640 1 -- 172.21.15.103:0/2794119076 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0019920 con 0x561cb28c1590 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.847+0000 7fcad7e00640 1 -- 172.21.15.103:0/1717048309 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcacc010ac0 con 0x7fcae01049c0 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.847+0000 7fcae9073740 1 -- 172.21.15.103:0/2794119076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 msgr2=0x561cb28cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.847+0000 7fcae9073740 1 --2- 172.21.15.103:0/2794119076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28cb910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009cf0 tx=0x7fcad0011810 comp rx=0 tx=0).stop 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.848+0000 7fcae9073740 1 -- 172.21.15.103:0/2794119076 shutdown_connections 2024-09-16T02:07:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.848+0000 7fcae9073740 1 --2- 172.21.15.103:0/2794119076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28cbe50 0x561cb28ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.848+0000 7fcae9073740 1 --2- 172.21.15.103:0/2794119076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28cb910 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.848+0000 7fcae9073740 1 --2- 172.21.15.103:0/2794119076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 0x561cb28c1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.848+0000 7fcae9073740 1 -- 172.21.15.103:0/2794119076 >> 172.21.15.103:0/2794119076 conn(0x561cb28bd200 msgr2=0x561cb28bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.848+0000 7fcae9073740 1 -- 172.21.15.103:0/2794119076 shutdown_connections 2024-09-16T02:07:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.848+0000 7fcae9073740 1 -- 172.21.15.103:0/2794119076 wait complete. 2024-09-16T02:07:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.848+0000 7fcad7e00640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcac40a31d0 0x7fcac40a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.849+0000 7fcad7e00640 1 -- 172.21.15.103:0/1717048309 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fcacc0cfbc0 con 0x7fcae01049c0 2024-09-16T02:07:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.849+0000 7fcae9073740 1 -- start start 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.849+0000 7fcadf400640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcac40a31d0 0x7fcac40a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.849+0000 7fcae9073740 1 Processor -- start 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.849+0000 7fcae9073740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28c0c70 0x561cb28c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcae9073740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 0x561cb28c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcae9073740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcae9073740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561cb28cea20 con 0x561cb28c1590 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcae9073740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cb27e20f0 con 0x561cb28cbe50 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcae9073740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561cb28ceba0 con 0x561cb28c0c70 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcae9073740 10 client.0 _ll_register_callbacks cb 0x561cb27c9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 0x561cb28c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadf400640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcac40a31d0 0x7fcac40a5690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcad001b040 tx=0x7fcad0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadf400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28c0c70 0x561cb28c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:47630/0 (socket says 172.21.15.103:47630) 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadfe00640 1 -- 172.21.15.103:0/4042262920 learned_addr learned my addr 172.21.15.103:0/4042262920 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadfe00640 1 -- 172.21.15.103:0/4042262920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28c0c70 msgr2=0x561cb28c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadfe00640 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28c0c70 0x561cb28c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadfe00640 1 -- 172.21.15.103:0/4042262920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 msgr2=0x561cb28c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadfe00640 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 0x561cb28c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.850+0000 7fcadfe00640 1 -- 172.21.15.103:0/4042262920 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb25983c0 con 0x561cb28cbe50 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.851+0000 7fcadfe00640 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c9810 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009ca0 tx=0x7fcad8003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=727f5a4c4406fd4e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.851+0000 7fcac9600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.851+0000 7fcac9600640 1 -- 172.21.15.103:0/4042262920 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad804a060 con 0x561cb28cbe50 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.851+0000 7fcac9600640 1 -- 172.21.15.103:0/4042262920 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fcad8031450 con 0x561cb28cbe50 2024-09-16T02:07:04.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.851+0000 7fcae9073740 1 -- 172.21.15.103:0/4042262920 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad00043d0 con 0x561cb28cbe50 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.851+0000 7fcac9600640 1 -- 172.21.15.103:0/4042262920 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcad8031650 con 0x561cb28cbe50 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.851+0000 7fcae9073740 10 client.25346 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.851+0000 7fcae9073740 1 -- 172.21.15.103:0/4042262920 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561cb28daac0 con 0x561cb28cbe50 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.852+0000 7fcac9600640 1 -- 172.21.15.103:0/4042262920 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad00043d0 con 0x561cb28cbe50 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.852+0000 7fcac9600640 1 -- 172.21.15.103:0/4042262920 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fcad802f7d0 con 0x561cb28cbe50 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.852+0000 7fcae9073740 10 client.25346 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.852+0000 7fcae9073740 10 client.25346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.852+0000 7fcae9073740 20 client.25346 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.852+0000 7fcae9073740 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb28dbf80 0x561cb28de390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.852+0000 7fcae9073740 4 client.25346 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.852+0000 7fcae9073740 1 -- 172.21.15.103:0/4042262920 --> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cb2834950 con 0x561cb28dbf80 2024-09-16T02:07:04.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.852+0000 7fcadf400640 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb28dbf80 0x561cb28de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.853+0000 7fcadf400640 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb28dbf80 0x561cb28de390 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89786bdadc0296dc server_cookie=a565cb72e68be27 in_seq=0 out_seq=0 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.853+0000 7fcac9600640 10 client.25346 ms_handle_connect on v2:172.21.15.172:6838/3474994563 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcac9600640 1 -- 172.21.15.103:0/4042262920 <== mds.2 v2:172.21.15.172:6838/3474994563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cb2834950 con 0x561cb28dbf80 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcac9600640 10 client.25346 handle_command_reply: tid=0 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 client.25346 shutdown 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 -- 172.21.15.103:0/4042262920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 msgr2=0x561cb28c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c9810 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009ca0 tx=0x7fcad8003f90 comp rx=0 tx=0).stop 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 -- 172.21.15.103:0/4042262920 shutdown_connections 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb28dbf80 0x561cb28de390 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c9810 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 0x561cb28c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 --2- 172.21.15.103:0/4042262920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28c0c70 0x561cb28c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 -- 172.21.15.103:0/4042262920 >> 172.21.15.103:0/4042262920 conn(0x561cb28bd200 msgr2=0x561cb28c2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcac9600640 0 client.25346 ms_handle_reset on v2:172.21.15.172:6838/3474994563 2024-09-16T02:07:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 -- 172.21.15.103:0/4042262920 shutdown_connections 2024-09-16T02:07:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 1 -- 172.21.15.103:0/4042262920 wait complete. 2024-09-16T02:07:04.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.854+0000 7fcae9073740 20 client.25346 trim_cache size 0 max 16384 2024-09-16T02:07:04.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.868+0000 7fcae9073740 1 Processor -- start 2024-09-16T02:07:04.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.868+0000 7fcae9073740 1 -- start start 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.868+0000 7fcae9073740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 0x561cb28e3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.868+0000 7fcae9073740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.868+0000 7fcae9073740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28cbe50 0x561cb28bbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.868+0000 7fcae9073740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01b2310 con 0x561cb28c0c70 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.868+0000 7fcae9073740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01afbe0 con 0x561cb28c1590 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.868+0000 7fcae9073740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01ada30 con 0x561cb28cbe50 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.868+0000 7fcadf400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 0x561cb28e3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadf400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 0x561cb28e3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52800/0 (socket says 172.21.15.103:52800) 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadf400640 1 -- 172.21.15.103:0/3714193218 learned_addr learned my addr 172.21.15.103:0/3714193218 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadea00640 1 --2- 172.21.15.103:0/3714193218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadfe00640 1 --2- 172.21.15.103:0/3714193218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28cbe50 0x561cb28bbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadf400640 1 -- 172.21.15.103:0/3714193218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28cbe50 msgr2=0x561cb28bbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:04.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadf400640 1 --2- 172.21.15.103:0/3714193218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28cbe50 0x561cb28bbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadf400640 1 -- 172.21.15.103:0/3714193218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 msgr2=0x561cb28e41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:04.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadf400640 1 --2- 172.21.15.103:0/3714193218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28e41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadf400640 1 -- 172.21.15.103:0/3714193218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb28b8770 con 0x561cb28c0c70 2024-09-16T02:07:04.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadea00640 1 --2- 172.21.15.103:0/3714193218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:04.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcadf400640 1 --2- 172.21.15.103:0/3714193218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 0x561cb28e3c60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcad001a000 tx=0x7fcad0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4073028f7f4bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:04.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.869+0000 7fcacbe00640 1 -- 172.21.15.103:0/3714193218 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad003d070 con 0x561cb28c0c70 2024-09-16T02:07:04.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcacbe00640 1 -- 172.21.15.103:0/3714193218 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad00054d0 con 0x561cb28c0c70 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcacbe00640 1 -- 172.21.15.103:0/3714193218 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0004e50 con 0x561cb28c0c70 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcae9073740 1 -- 172.21.15.103:0/3714193218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 msgr2=0x561cb28e3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcae9073740 1 --2- 172.21.15.103:0/3714193218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 0x561cb28e3c60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcad001a000 tx=0x7fcad0005f90 comp rx=0 tx=0).stop 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcae9073740 1 -- 172.21.15.103:0/3714193218 shutdown_connections 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcae9073740 1 --2- 172.21.15.103:0/3714193218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28cbe50 0x561cb28bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcae9073740 1 --2- 172.21.15.103:0/3714193218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28c1590 0x561cb28e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcae9073740 1 --2- 172.21.15.103:0/3714193218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c0c70 0x561cb28e3c60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcae9073740 1 -- 172.21.15.103:0/3714193218 >> 172.21.15.103:0/3714193218 conn(0x561cb28bd200 msgr2=0x561cb28c4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcae9073740 1 -- 172.21.15.103:0/3714193218 shutdown_connections 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.870+0000 7fcae9073740 1 -- 172.21.15.103:0/3714193218 wait complete. 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.871+0000 7fcae9073740 1 -- start start 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.871+0000 7fcae9073740 1 Processor -- start 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.871+0000 7fcae9073740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28c0c70 0x561cb28d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.871+0000 7fcae9073740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 0x561cb28c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.871+0000 7fcae9073740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.871+0000 7fcae9073740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01ae0a0 con 0x561cb28c1590 2024-09-16T02:07:04.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.871+0000 7fcae9073740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01af510 con 0x561cb28cbe50 2024-09-16T02:07:04.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.871+0000 7fcae9073740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01ae770 con 0x561cb28c0c70 2024-09-16T02:07:04.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcae9073740 10 client.0 _ll_register_callbacks cb 0x7fcad8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:04.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 0x561cb28c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 0x561cb28c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52808/0 (socket says 172.21.15.103:52808) 2024-09-16T02:07:04.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadea00640 1 -- 172.21.15.103:0/3442465202 learned_addr learned my addr 172.21.15.103:0/3442465202 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadf400640 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28c0c70 0x561cb28d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadfe00640 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadea00640 1 -- 172.21.15.103:0/3442465202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28c0c70 msgr2=0x561cb28d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadea00640 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28c0c70 0x561cb28d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadea00640 1 -- 172.21.15.103:0/3442465202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 msgr2=0x561cb28c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadea00640 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadea00640 1 -- 172.21.15.103:0/3442465202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb28c6400 con 0x561cb28c1590 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.872+0000 7fcadfe00640 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.873+0000 7fcadea00640 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 0x561cb28c58d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcacc098b80 tx=0x7fcacc0a04b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c533d9b289f67d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.873+0000 7fcac9600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.873+0000 7fcac9600640 1 -- 172.21.15.103:0/3442465202 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0bc0b0 con 0x561cb28c1590 2024-09-16T02:07:04.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.873+0000 7fcae9073740 1 -- 172.21.15.103:0/3442465202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad00054d0 con 0x561cb28c1590 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.873+0000 7fcac9600640 1 -- 172.21.15.103:0/3442465202 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fcacc018b30 con 0x561cb28c1590 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.873+0000 7fcac9600640 1 -- 172.21.15.103:0/3442465202 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcacc018d30 con 0x561cb28c1590 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.873+0000 7fcae9073740 10 client.15387 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.873+0000 7fcae9073740 1 -- 172.21.15.103:0/3442465202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561cb2946a30 con 0x561cb28c1590 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.873+0000 7fcac9600640 1 -- 172.21.15.103:0/3442465202 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad00054d0 con 0x561cb28c1590 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.874+0000 7fcac9600640 1 -- 172.21.15.103:0/3442465202 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fcacc098020 con 0x561cb28c1590 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.874+0000 7fcae9073740 10 client.15387 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.874+0000 7fcae9073740 10 client.15387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.874+0000 7fcae9073740 20 client.15387 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.874+0000 7fcae9073740 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb2947230 0x561cb2949650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.874+0000 7fcae9073740 4 client.15387 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.874+0000 7fcae9073740 1 -- 172.21.15.103:0/3442465202 --> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cb282ee60 con 0x561cb2947230 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.874+0000 7fcadf400640 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb2947230 0x561cb2949650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:04.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.874+0000 7fcadf400640 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb2947230 0x561cb2949650 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1353f06d8a84485 server_cookie=287bfc82260e3067 in_seq=0 out_seq=0 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.874+0000 7fcac9600640 10 client.15387 ms_handle_connect on v2:172.21.15.172:6838/3474994563 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcac9600640 1 -- 172.21.15.103:0/3442465202 <== mds.2 v2:172.21.15.172:6838/3474994563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cb282ee60 con 0x561cb2947230 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcac9600640 10 client.15387 handle_command_reply: tid=0 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 client.15387 shutdown 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 -- 172.21.15.103:0/3442465202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 msgr2=0x561cb28c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 0x561cb28c58d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcacc098b80 tx=0x7fcacc0a04b0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 -- 172.21.15.103:0/3442465202 shutdown_connections 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x561cb2947230 0x561cb2949650 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cb28cbe50 0x561cb28c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cb28c1590 0x561cb28c58d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcac9600640 0 client.15387 ms_handle_reset on v2:172.21.15.172:6838/3474994563 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 --2- 172.21.15.103:0/3442465202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561cb28c0c70 0x561cb28d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 -- 172.21.15.103:0/3442465202 >> 172.21.15.103:0/3442465202 conn(0x561cb28bd200 msgr2=0x561cb28ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:04.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 -- 172.21.15.103:0/3442465202 shutdown_connections 2024-09-16T02:07:04.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 1 -- 172.21.15.103:0/3442465202 wait complete. 2024-09-16T02:07:04.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.875+0000 7fcae9073740 20 client.15387 trim_cache size 0 max 16384 2024-09-16T02:07:04.881 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 -- 172.21.15.103:0/1717048309 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcac40a31d0 msgr2=0x7fcac40a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcac40a31d0 0x7fcac40a5690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcad001b040 tx=0x7fcad0000bc0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 -- 172.21.15.103:0/1717048309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae01049c0 msgr2=0x7fcae019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae01049c0 0x7fcae019abd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00e9f0 tx=0x7fcacc00eeb0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 -- 172.21.15.103:0/1717048309 shutdown_connections 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fcac40a31d0 0x7fcac40a5690 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae0109140 0x7fcae019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcae01049c0 0x7fcae019abd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 --2- 172.21.15.103:0/1717048309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae0104010 0x7fcae019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 -- 172.21.15.103:0/1717048309 >> 172.21.15.103:0/1717048309 conn(0x7fcae00ffa50 msgr2=0x7fcae0100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 -- 172.21.15.103:0/1717048309 shutdown_connections 2024-09-16T02:07:04.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:04.876+0000 7fcae6600640 1 -- 172.21.15.103:0/1717048309 wait complete. 2024-09-16T02:07:05.600 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T02:07:05.882 INFO:tasks.workunit.client.0.smithi103.stdout:writing pattern 2024-09-16T02:07:05.882 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-09-16T02:07:05.882 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-16T02:07:05.882 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-09-16T02:07:05.882 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-16T02:07:05.882 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-09-16T02:07:05.882 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-09-16T02:07:05.883 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-09-16T02:07:05.884 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-16T02:07:05.885 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-16T02:07:05.886 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-09-16T02:07:05.887 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-16T02:07:05.888 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-09-16T02:07:05.912 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-16T02:07:05.912 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-09-16T02:07:05.912 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-16T02:07:05.912 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-09-16T02:07:05.912 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-16T02:07:05.912 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-09-16T02:07:05.913 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-16T02:07:05.913 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-09-16T02:07:05.913 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-16T02:07:05.913 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-09-16T02:07:05.913 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-16T02:07:05.913 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-09-16T02:07:05.913 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-16T02:07:05.913 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-09-16T02:07:05.913 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-09-16T02:07:05.914 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-16T02:07:05.915 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-09-16T02:07:05.915 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-16T02:07:05.915 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-09-16T02:07:05.915 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-16T02:07:05.915 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-09-16T02:07:05.915 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-16T02:07:05.915 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-09-16T02:07:05.915 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-16T02:07:05.915 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-09-16T02:07:05.915 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-16T02:07:05.916 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-09-16T02:07:05.916 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-16T02:07:05.916 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-09-16T02:07:05.916 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-16T02:07:05.916 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-09-16T02:07:05.916 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-16T02:07:05.916 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-09-16T02:07:05.916 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-16T02:07:05.916 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-09-16T02:07:05.917 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-16T02:07:05.917 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-09-16T02:07:05.917 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-16T02:07:05.917 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-09-16T02:07:05.917 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-16T02:07:05.917 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-09-16T02:07:05.917 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-16T02:07:05.917 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-09-16T02:07:05.917 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-16T02:07:05.917 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-09-16T02:07:05.918 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-16T02:07:05.918 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-09-16T02:07:05.918 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-16T02:07:05.918 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-09-16T02:07:05.918 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-16T02:07:05.918 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-09-16T02:07:05.919 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-16T02:07:05.919 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-09-16T02:07:05.919 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-16T02:07:05.919 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-09-16T02:07:05.919 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-16T02:07:05.919 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-09-16T02:07:05.920 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-16T02:07:05.920 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-09-16T02:07:05.920 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-16T02:07:05.920 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-09-16T02:07:05.920 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-16T02:07:05.920 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-09-16T02:07:05.920 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-16T02:07:05.920 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-09-16T02:07:05.920 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-16T02:07:05.920 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-09-16T02:07:05.921 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-16T02:07:05.921 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-09-16T02:07:05.921 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-16T02:07:05.921 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-09-16T02:07:05.921 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-16T02:07:05.921 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-09-16T02:07:05.921 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-16T02:07:05.921 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-09-16T02:07:05.929 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-16T02:07:05.929 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-09-16T02:07:05.929 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-16T02:07:05.929 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-09-16T02:07:05.929 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-16T02:07:05.929 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-09-16T02:07:05.929 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-16T02:07:05.929 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-09-16T02:07:05.929 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-16T02:07:05.929 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-09-16T02:07:05.930 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-16T02:07:05.931 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-16T02:07:05.932 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-16T02:07:05.933 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-09-16T02:07:05.934 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-16T02:07:05.935 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-09-16T02:07:05.935 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-16T02:07:05.935 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-09-16T02:07:05.935 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-16T02:07:05.935 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-09-16T02:07:05.935 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-16T02:07:05.935 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-09-16T02:07:05.935 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-16T02:07:05.935 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-09-16T02:07:05.935 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-16T02:07:05.948 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-09-16T02:07:05.949 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-16T02:07:05.949 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-09-16T02:07:05.949 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-16T02:07:05.949 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-09-16T02:07:05.949 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-16T02:07:05.949 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-09-16T02:07:05.949 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-16T02:07:05.949 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-09-16T02:07:05.950 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-09-16T02:07:05.951 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-16T02:07:05.952 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-09-16T02:07:05.952 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-16T02:07:05.952 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-09-16T02:07:05.952 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-16T02:07:05.952 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-09-16T02:07:05.952 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-16T02:07:05.953 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-16T02:07:05.954 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-16T02:07:05.955 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-09-16T02:07:05.956 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-16T02:07:05.956 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-09-16T02:07:05.956 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-16T02:07:05.956 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-09-16T02:07:05.956 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-16T02:07:05.956 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-09-16T02:07:05.956 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-16T02:07:05.956 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-09-16T02:07:05.956 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-16T02:07:05.957 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-09-16T02:07:05.977 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-16T02:07:05.977 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-09-16T02:07:05.977 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-16T02:07:05.977 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-09-16T02:07:05.977 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-16T02:07:05.977 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-09-16T02:07:05.978 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-09-16T02:07:05.979 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-16T02:07:05.980 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-09-16T02:07:05.980 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-16T02:07:05.980 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-09-16T02:07:05.980 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-16T02:07:05.980 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-09-16T02:07:05.980 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-16T02:07:05.980 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-09-16T02:07:05.980 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-16T02:07:05.980 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-09-16T02:07:05.981 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-16T02:07:05.982 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-16T02:07:05.983 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-09-16T02:07:05.984 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-16T02:07:05.984 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-09-16T02:07:05.984 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-16T02:07:05.984 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-09-16T02:07:05.984 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-16T02:07:05.984 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-09-16T02:07:05.984 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-16T02:07:05.984 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-09-16T02:07:06.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:05 smithi103 ceph-mon[30398]: pgmap v389: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 386 KiB/s wr, 2.29k op/s 2024-09-16T02:07:06.007 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-16T02:07:06.007 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-09-16T02:07:06.007 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-16T02:07:06.007 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-09-16T02:07:06.008 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-16T02:07:06.008 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-09-16T02:07:06.008 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-16T02:07:06.008 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-09-16T02:07:06.008 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-16T02:07:06.008 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-09-16T02:07:06.009 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-16T02:07:06.009 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-09-16T02:07:06.009 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-16T02:07:06.009 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-09-16T02:07:06.009 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-16T02:07:06.010 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-09-16T02:07:06.010 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-16T02:07:06.010 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-09-16T02:07:06.010 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-16T02:07:06.010 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-09-16T02:07:06.010 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-16T02:07:06.011 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-09-16T02:07:06.011 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-16T02:07:06.011 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-09-16T02:07:06.011 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-16T02:07:06.012 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-09-16T02:07:06.012 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-16T02:07:06.012 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-09-16T02:07:06.012 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-16T02:07:06.012 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-09-16T02:07:06.012 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-16T02:07:06.012 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-16T02:07:06.013 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-09-16T02:07:06.014 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-16T02:07:06.015 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-09-16T02:07:06.016 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-16T02:07:06.016 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-09-16T02:07:06.016 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-16T02:07:06.016 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-09-16T02:07:06.016 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-16T02:07:06.016 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-09-16T02:07:06.016 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-16T02:07:06.016 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-09-16T02:07:06.016 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-16T02:07:06.017 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-09-16T02:07:06.017 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-16T02:07:06.017 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-09-16T02:07:06.017 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-16T02:07:06.017 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-09-16T02:07:06.017 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:06.020 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-16T02:07:06.020 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-09-16T02:07:06.020 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-16T02:07:06.021 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-09-16T02:07:06.021 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-16T02:07:06.021 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-09-16T02:07:06.021 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-16T02:07:06.021 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-09-16T02:07:06.021 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-16T02:07:06.021 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-09-16T02:07:06.021 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-16T02:07:06.021 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-09-16T02:07:06.022 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-16T02:07:06.023 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-09-16T02:07:06.023 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-16T02:07:06.023 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-09-16T02:07:06.023 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-16T02:07:06.023 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-09-16T02:07:06.023 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-16T02:07:06.023 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-09-16T02:07:06.023 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-16T02:07:06.023 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-09-16T02:07:06.024 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-16T02:07:06.024 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-09-16T02:07:06.024 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-16T02:07:06.024 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-09-16T02:07:06.024 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-16T02:07:06.024 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-09-16T02:07:06.024 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-16T02:07:06.024 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-09-16T02:07:06.024 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-16T02:07:06.025 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-09-16T02:07:06.025 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-16T02:07:06.025 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-09-16T02:07:06.025 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-16T02:07:06.025 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-09-16T02:07:06.025 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-16T02:07:06.025 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-16T02:07:06.026 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-09-16T02:07:06.027 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-16T02:07:06.027 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-09-16T02:07:06.027 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-16T02:07:06.027 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-09-16T02:07:06.027 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-16T02:07:06.027 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-09-16T02:07:06.027 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-16T02:07:06.027 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-09-16T02:07:06.027 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-16T02:07:06.028 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-09-16T02:07:06.029 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-16T02:07:06.029 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-09-16T02:07:06.029 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:05 smithi136 ceph-mon[35323]: pgmap v389: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 386 KiB/s wr, 2.29k op/s 2024-09-16T02:07:06.039 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-16T02:07:06.039 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-09-16T02:07:06.039 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-16T02:07:06.039 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-09-16T02:07:06.039 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-16T02:07:06.039 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-09-16T02:07:06.039 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-16T02:07:06.039 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-09-16T02:07:06.039 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-16T02:07:06.040 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-09-16T02:07:06.041 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-16T02:07:06.041 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-09-16T02:07:06.041 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-16T02:07:06.041 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-09-16T02:07:06.041 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-16T02:07:06.041 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-09-16T02:07:06.041 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-16T02:07:06.041 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-09-16T02:07:06.041 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-16T02:07:06.041 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-09-16T02:07:06.042 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-16T02:07:06.042 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-09-16T02:07:06.042 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-16T02:07:06.042 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-09-16T02:07:06.042 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-16T02:07:06.042 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-09-16T02:07:06.042 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-16T02:07:06.043 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-09-16T02:07:06.043 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-16T02:07:06.043 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-09-16T02:07:06.043 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-16T02:07:06.043 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-09-16T02:07:06.043 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-16T02:07:06.043 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-09-16T02:07:06.043 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-16T02:07:06.043 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-09-16T02:07:06.044 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-16T02:07:06.045 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-09-16T02:07:06.045 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-16T02:07:06.045 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-09-16T02:07:06.045 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-16T02:07:06.045 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-09-16T02:07:06.045 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-16T02:07:06.045 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-09-16T02:07:06.045 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-16T02:07:06.046 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-09-16T02:07:06.047 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-16T02:07:06.047 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:05 smithi172 ceph-mon[35340]: pgmap v389: 97 pgs: 97 active+clean; 1.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 386 KiB/s wr, 2.29k op/s 2024-09-16T02:07:06.070 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-09-16T02:07:06.070 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-16T02:07:06.071 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-09-16T02:07:06.071 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-16T02:07:06.071 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-09-16T02:07:06.071 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-16T02:07:06.071 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-09-16T02:07:06.071 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-16T02:07:06.071 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-09-16T02:07:06.071 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-16T02:07:06.072 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-09-16T02:07:06.072 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-16T02:07:06.072 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-09-16T02:07:06.072 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-16T02:07:06.072 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-09-16T02:07:06.072 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-16T02:07:06.072 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-09-16T02:07:06.072 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-16T02:07:06.072 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-09-16T02:07:06.072 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-16T02:07:06.073 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-09-16T02:07:06.073 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-16T02:07:06.073 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-09-16T02:07:06.073 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-16T02:07:06.073 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-09-16T02:07:06.073 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-16T02:07:06.073 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-09-16T02:07:06.073 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-16T02:07:06.073 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-09-16T02:07:06.073 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-16T02:07:06.074 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-09-16T02:07:06.075 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-16T02:07:06.076 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-09-16T02:07:06.077 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-16T02:07:06.077 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-09-16T02:07:06.077 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-16T02:07:06.077 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-09-16T02:07:06.077 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-16T02:07:06.077 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-09-16T02:07:06.077 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-16T02:07:06.077 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-09-16T02:07:06.077 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-16T02:07:06.077 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-09-16T02:07:06.095 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-16T02:07:06.095 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-09-16T02:07:06.096 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-09-16T02:07:06.097 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-09-16T02:07:06.098 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-16T02:07:06.099 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-09-16T02:07:06.099 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-16T02:07:06.099 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-09-16T02:07:06.099 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-16T02:07:06.099 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-09-16T02:07:06.099 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-16T02:07:06.099 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-09-16T02:07:06.099 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-16T02:07:06.099 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-09-16T02:07:06.100 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-16T02:07:06.101 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-09-16T02:07:06.101 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-16T02:07:06.101 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-09-16T02:07:06.101 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-16T02:07:06.101 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-09-16T02:07:06.101 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-16T02:07:06.101 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-09-16T02:07:06.101 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-16T02:07:06.102 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-09-16T02:07:06.102 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-16T02:07:06.102 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-09-16T02:07:06.102 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-16T02:07:06.102 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-09-16T02:07:06.102 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-16T02:07:06.102 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-09-16T02:07:06.102 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-16T02:07:06.102 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-09-16T02:07:06.107 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-16T02:07:06.107 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-09-16T02:07:06.107 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-16T02:07:06.107 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-09-16T02:07:06.107 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-16T02:07:06.107 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-09-16T02:07:06.107 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-16T02:07:06.108 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-09-16T02:07:06.108 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-16T02:07:06.108 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-09-16T02:07:06.108 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-16T02:07:06.108 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-09-16T02:07:06.108 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-16T02:07:06.108 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-09-16T02:07:06.109 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-16T02:07:06.109 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-09-16T02:07:06.109 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-16T02:07:06.109 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-09-16T02:07:06.109 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-16T02:07:06.109 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-09-16T02:07:06.109 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-09-16T02:07:06.110 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-16T02:07:06.111 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-09-16T02:07:06.111 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-16T02:07:06.111 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-09-16T02:07:06.111 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-16T02:07:06.111 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-09-16T02:07:06.111 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-16T02:07:06.111 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-09-16T02:07:06.111 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-16T02:07:06.111 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-09-16T02:07:06.111 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-16T02:07:06.112 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-16T02:07:06.113 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-09-16T02:07:06.114 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-16T02:07:06.131 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-09-16T02:07:06.131 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-16T02:07:06.131 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-09-16T02:07:06.131 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-16T02:07:06.131 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-09-16T02:07:06.131 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-16T02:07:06.131 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-09-16T02:07:06.131 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-16T02:07:06.132 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-09-16T02:07:06.133 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-16T02:07:06.133 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-09-16T02:07:06.133 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-16T02:07:06.133 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-09-16T02:07:06.133 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-16T02:07:06.133 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-09-16T02:07:06.133 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-16T02:07:06.133 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-09-16T02:07:06.134 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-16T02:07:06.135 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-09-16T02:07:06.135 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-16T02:07:06.135 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-09-16T02:07:06.135 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-16T02:07:06.135 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-09-16T02:07:06.135 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-16T02:07:06.135 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-09-16T02:07:06.135 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-16T02:07:06.136 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-09-16T02:07:06.137 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-16T02:07:06.137 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-09-16T02:07:06.137 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-16T02:07:06.137 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-09-16T02:07:06.137 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-16T02:07:06.137 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-09-16T02:07:06.137 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-16T02:07:06.137 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-09-16T02:07:06.138 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-16T02:07:06.138 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-09-16T02:07:06.138 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-16T02:07:06.138 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-09-16T02:07:06.138 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-16T02:07:06.138 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-09-16T02:07:06.138 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-16T02:07:06.138 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-09-16T02:07:06.138 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-16T02:07:06.139 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-09-16T02:07:06.139 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-16T02:07:06.139 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-09-16T02:07:06.139 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-16T02:07:06.162 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-09-16T02:07:06.163 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-16T02:07:06.164 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-09-16T02:07:06.165 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-16T02:07:06.165 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-09-16T02:07:06.165 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-16T02:07:06.165 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-09-16T02:07:06.165 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-16T02:07:06.165 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-09-16T02:07:06.165 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-16T02:07:06.165 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-16T02:07:06.166 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-09-16T02:07:06.167 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-16T02:07:06.167 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-09-16T02:07:06.167 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-16T02:07:06.167 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-09-16T02:07:06.167 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-16T02:07:06.167 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-09-16T02:07:06.167 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-16T02:07:06.167 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-16T02:07:06.168 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-09-16T02:07:06.169 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-16T02:07:06.169 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-09-16T02:07:06.169 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-16T02:07:06.169 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-09-16T02:07:06.169 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-16T02:07:06.169 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-09-16T02:07:06.169 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-16T02:07:06.169 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-09-16T02:07:06.170 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-16T02:07:06.170 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-09-16T02:07:06.170 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-16T02:07:06.170 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-09-16T02:07:06.170 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-16T02:07:06.170 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-09-16T02:07:06.170 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-16T02:07:06.170 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-09-16T02:07:06.170 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-16T02:07:06.181 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-09-16T02:07:06.181 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-16T02:07:06.181 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-09-16T02:07:06.181 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-16T02:07:06.181 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-09-16T02:07:06.181 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-16T02:07:06.182 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-09-16T02:07:06.182 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-16T02:07:06.182 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-09-16T02:07:06.182 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-16T02:07:06.182 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-09-16T02:07:06.182 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-16T02:07:06.182 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-09-16T02:07:06.183 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-09-16T02:07:06.184 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-16T02:07:06.185 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-09-16T02:07:06.186 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-09-16T02:07:06.187 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-16T02:07:06.188 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-09-16T02:07:06.188 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-16T02:07:06.188 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-09-16T02:07:06.193 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-09-16T02:07:06.194 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-09-16T02:07:06.195 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-09-16T02:07:06.196 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-16T02:07:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-09-16T02:07:06.198 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-16T02:07:06.199 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-09-16T02:07:06.221 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-16T02:07:06.221 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-09-16T02:07:06.221 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-16T02:07:06.222 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-09-16T02:07:06.223 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-16T02:07:06.224 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-09-16T02:07:06.224 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-16T02:07:06.224 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-09-16T02:07:06.224 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-16T02:07:06.224 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-09-16T02:07:06.224 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-16T02:07:06.224 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-09-16T02:07:06.224 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-16T02:07:06.224 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-09-16T02:07:06.225 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-16T02:07:06.225 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-09-16T02:07:06.225 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-16T02:07:06.225 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-09-16T02:07:06.225 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-16T02:07:06.225 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-09-16T02:07:06.225 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-16T02:07:06.226 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-09-16T02:07:06.226 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-16T02:07:06.226 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-09-16T02:07:06.226 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-16T02:07:06.226 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-09-16T02:07:06.226 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-16T02:07:06.226 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-09-16T02:07:06.226 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-16T02:07:06.226 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-09-16T02:07:06.226 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-16T02:07:06.227 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-09-16T02:07:06.228 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-09-16T02:07:06.251 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-09-16T02:07:06.252 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-09-16T02:07:06.253 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-09-16T02:07:06.254 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-09-16T02:07:06.255 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-16T02:07:06.256 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-09-16T02:07:06.257 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-16T02:07:06.257 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-09-16T02:07:06.257 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-16T02:07:06.266 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-09-16T02:07:06.266 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-16T02:07:06.266 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-09-16T02:07:06.266 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-09-16T02:07:06.267 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-09-16T02:07:06.268 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-16T02:07:06.269 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-09-16T02:07:06.270 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-09-16T02:07:06.271 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-16T02:07:06.272 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-09-16T02:07:06.272 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-16T02:07:06.272 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-09-16T02:07:06.272 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-16T02:07:06.272 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-09-16T02:07:06.272 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-16T02:07:06.272 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-09-16T02:07:06.272 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-16T02:07:06.272 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-09-16T02:07:06.272 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-16T02:07:06.279 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-09-16T02:07:06.279 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-16T02:07:06.279 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-09-16T02:07:06.279 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-16T02:07:06.279 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-09-16T02:07:06.279 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-09-16T02:07:06.280 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-09-16T02:07:06.281 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-09-16T02:07:06.282 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-16T02:07:06.283 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-16T02:07:06.284 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-09-16T02:07:06.285 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-16T02:07:06.285 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-09-16T02:07:06.285 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-16T02:07:06.285 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-09-16T02:07:06.285 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-16T02:07:06.285 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-09-16T02:07:06.306 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-16T02:07:06.306 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-09-16T02:07:06.306 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-16T02:07:06.306 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-09-16T02:07:06.306 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-16T02:07:06.306 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-16T02:07:06.307 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-09-16T02:07:06.308 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-16T02:07:06.309 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-09-16T02:07:06.310 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-16T02:07:06.311 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-09-16T02:07:06.312 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-16T02:07:06.312 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-09-16T02:07:06.312 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-16T02:07:06.312 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-09-16T02:07:06.346 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-16T02:07:06.346 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-09-16T02:07:06.346 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-16T02:07:06.346 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-09-16T02:07:06.346 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-16T02:07:06.346 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-09-16T02:07:06.346 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-16T02:07:06.346 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-09-16T02:07:06.347 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-09-16T02:07:06.348 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-09-16T02:07:06.349 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-16T02:07:06.350 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-16T02:07:06.351 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-09-16T02:07:06.352 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-16T02:07:06.352 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-09-16T02:07:06.352 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-16T02:07:06.352 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-09-16T02:07:06.352 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-16T02:07:06.352 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-09-16T02:07:06.352 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-16T02:07:06.360 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-09-16T02:07:06.361 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-09-16T02:07:06.362 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-09-16T02:07:06.363 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-09-16T02:07:06.364 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-16T02:07:06.365 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-09-16T02:07:06.366 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-16T02:07:06.372 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-09-16T02:07:06.372 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-16T02:07:06.372 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-09-16T02:07:06.372 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-16T02:07:06.372 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-09-16T02:07:06.373 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-16T02:07:06.374 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-16T02:07:06.375 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-16T02:07:06.376 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-16T02:07:06.377 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-09-16T02:07:06.378 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-16T02:07:06.378 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-09-16T02:07:06.378 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-16T02:07:06.378 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-09-16T02:07:06.378 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-16T02:07:06.378 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-09-16T02:07:06.378 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-16T02:07:06.399 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-16T02:07:06.400 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-09-16T02:07:06.401 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-09-16T02:07:06.402 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-09-16T02:07:06.403 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-09-16T02:07:06.404 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-16T02:07:06.405 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-16T02:07:06.431 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-09-16T02:07:06.432 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-09-16T02:07:06.433 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-16T02:07:06.434 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-09-16T02:07:06.435 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-16T02:07:06.436 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-09-16T02:07:06.437 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-16T02:07:06.437 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-09-16T02:07:06.443 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-16T02:07:06.443 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-09-16T02:07:06.443 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-16T02:07:06.443 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-09-16T02:07:06.443 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-16T02:07:06.443 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-09-16T02:07:06.443 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-16T02:07:06.444 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-16T02:07:06.445 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-09-16T02:07:06.446 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-16T02:07:06.447 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-16T02:07:06.448 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-09-16T02:07:06.449 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-16T02:07:06.449 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-09-16T02:07:06.449 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-16T02:07:06.449 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-09-16T02:07:06.449 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-16T02:07:06.449 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-09-16T02:07:06.449 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-16T02:07:06.449 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-09-16T02:07:06.449 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-16T02:07:06.455 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-09-16T02:07:06.455 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-16T02:07:06.455 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-16T02:07:06.456 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-16T02:07:06.457 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-16T02:07:06.458 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-16T02:07:06.459 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-16T02:07:06.460 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-09-16T02:07:06.461 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-16T02:07:06.461 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-09-16T02:07:06.461 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-16T02:07:06.461 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-09-16T02:07:06.461 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-16T02:07:06.461 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-09-16T02:07:06.461 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-16T02:07:06.461 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-09-16T02:07:06.461 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-16T02:07:06.485 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-09-16T02:07:06.485 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-16T02:07:06.486 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-09-16T02:07:06.487 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-16T02:07:06.488 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-16T02:07:06.489 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-16T02:07:06.490 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-09-16T02:07:06.491 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-16T02:07:06.492 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-09-16T02:07:06.512 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-16T02:07:06.512 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-09-16T02:07:06.513 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-09-16T02:07:06.514 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-16T02:07:06.515 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-09-16T02:07:06.516 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-16T02:07:06.516 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-09-16T02:07:06.516 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-16T02:07:06.516 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-09-16T02:07:06.516 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-16T02:07:06.516 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-09-16T02:07:06.516 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-16T02:07:06.516 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-09-16T02:07:06.516 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-16T02:07:06.516 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-09-16T02:07:06.517 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-09-16T02:07:06.518 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-16T02:07:06.519 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-09-16T02:07:06.519 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-16T02:07:06.519 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-16T02:07:06.525 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-09-16T02:07:06.526 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-16T02:07:06.526 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-09-16T02:07:06.526 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-16T02:07:06.526 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-09-16T02:07:06.526 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-16T02:07:06.526 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-09-16T02:07:06.526 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-16T02:07:06.526 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-09-16T02:07:06.526 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-16T02:07:06.526 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-09-16T02:07:06.527 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-16T02:07:06.528 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-09-16T02:07:06.529 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-16T02:07:06.530 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-09-16T02:07:06.531 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-16T02:07:06.531 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-09-16T02:07:06.531 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-16T02:07:06.531 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-09-16T02:07:06.531 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-16T02:07:06.531 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-09-16T02:07:06.531 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-16T02:07:06.531 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-09-16T02:07:06.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.600+0000 7f1a9b400640 1 -- 172.21.15.103:0/125406133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a94104010 msgr2=0x7f1a941043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:06.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.600+0000 7f1a9b400640 1 --2- 172.21.15.103:0/125406133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a94104010 0x7f1a941043f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009900 tx=0x7f1a8802a3f0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.601+0000 7f1a9b400640 1 -- 172.21.15.103:0/125406133 shutdown_connections 2024-09-16T02:07:06.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.601+0000 7f1a9b400640 1 --2- 172.21.15.103:0/125406133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a94109140 0x7f1a94110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.601+0000 7f1a9b400640 1 --2- 172.21.15.103:0/125406133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a941049c0 0x7f1a94108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.601+0000 7f1a9b400640 1 --2- 172.21.15.103:0/125406133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a94104010 0x7f1a941043f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.601+0000 7f1a9b400640 1 -- 172.21.15.103:0/125406133 >> 172.21.15.103:0/125406133 conn(0x7f1a940ffa50 msgr2=0x7f1a94101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:06.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.601+0000 7f1a9b400640 1 -- 172.21.15.103:0/125406133 shutdown_connections 2024-09-16T02:07:06.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.601+0000 7f1a9b400640 1 -- 172.21.15.103:0/125406133 wait complete. 2024-09-16T02:07:06.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.602+0000 7f1a9b400640 1 Processor -- start 2024-09-16T02:07:06.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.602+0000 7f1a9b400640 1 -- start start 2024-09-16T02:07:06.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a9b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a94104010 0x7f1a9419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a9b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a941049c0 0x7f1a9419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a9b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a94109140 0x7f1a9419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a9b400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941141c0 con 0x7f1a941049c0 2024-09-16T02:07:06.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a9b400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94114040 con 0x7f1a94104010 2024-09-16T02:07:06.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a9b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94114340 con 0x7f1a94109140 2024-09-16T02:07:06.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a941049c0 0x7f1a9419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a94104010 0x7f1a9419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a941049c0 0x7f1a9419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52822/0 (socket says 172.21.15.103:52822) 2024-09-16T02:07:06.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a93e00640 1 -- 172.21.15.103:0/4288667138 learned_addr learned my addr 172.21.15.103:0/4288667138 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:06.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a99400640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a94109140 0x7f1a9419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a94104010 0x7f1a9419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:47666/0 (socket says 172.21.15.103:47666) 2024-09-16T02:07:06.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a93e00640 1 -- 172.21.15.103:0/4288667138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a94109140 msgr2=0x7f1a9419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:06.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a93e00640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a94109140 0x7f1a9419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a93e00640 1 -- 172.21.15.103:0/4288667138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a94104010 msgr2=0x7f1a9419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:06.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a93e00640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a94104010 0x7f1a9419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a93e00640 1 -- 172.21.15.103:0/4288667138 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9419f5f0 con 0x7f1a941049c0 2024-09-16T02:07:06.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.603+0000 7f1a98a00640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a94104010 0x7f1a9419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:06.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.604+0000 7f1a93e00640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a941049c0 0x7f1a9419ab80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000ea60 tx=0x7f1a8000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:06.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.604+0000 7f1a91600640 1 -- 172.21.15.103:0/4288667138 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000cde0 con 0x7f1a941049c0 2024-09-16T02:07:06.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.604+0000 7f1a9b400640 1 -- 172.21.15.103:0/4288667138 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9419f8e0 con 0x7f1a941049c0 2024-09-16T02:07:06.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.604+0000 7f1a91600640 1 -- 172.21.15.103:0/4288667138 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a800035f0 con 0x7f1a941049c0 2024-09-16T02:07:06.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.604+0000 7f1a91600640 1 -- 172.21.15.103:0/4288667138 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80010670 con 0x7f1a941049c0 2024-09-16T02:07:06.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.604+0000 7f1a9b400640 1 -- 172.21.15.103:0/4288667138 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a941a71c0 con 0x7f1a941049c0 2024-09-16T02:07:06.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.608+0000 7f1a9de9e740 1 Processor -- start 2024-09-16T02:07:06.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.608+0000 7f1a9de9e740 1 -- start start 2024-09-16T02:07:06.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.608+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5aec70 0x557d1d5af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.608+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5b9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.608+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5b9e50 0x557d1d5bc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.608+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d1d5153e0 con 0x557d1d5b9e50 2024-09-16T02:07:06.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.608+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557d1d4bcd80 con 0x557d1d5aec70 2024-09-16T02:07:06.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.608+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557d1d5113a0 con 0x557d1d5af590 2024-09-16T02:07:06.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5b9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:52166/0 (socket says 172.21.15.103:52166) 2024-09-16T02:07:06.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a98a00640 1 -- 172.21.15.103:0/292460876 learned_addr learned my addr 172.21.15.103:0/292460876 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:06.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a99400640 1 --2- 172.21.15.103:0/292460876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5aec70 0x557d1d5af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a98a00640 1 -- 172.21.15.103:0/292460876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5aec70 msgr2=0x557d1d5af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:06.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a98a00640 1 --2- 172.21.15.103:0/292460876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5aec70 0x557d1d5af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a98a00640 1 -- 172.21.15.103:0/292460876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5b9e50 msgr2=0x557d1d5bc240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T02:07:06.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a98a00640 1 --2- 172.21.15.103:0/292460876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5b9e50 0x557d1d5bc240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a98a00640 1 -- 172.21.15.103:0/292460876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1d4dbae0 con 0x557d1d5af590 2024-09-16T02:07:06.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a99400640 1 --2- 172.21.15.103:0/292460876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5aec70 0x557d1d5af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:06.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.609+0000 7f1a98a00640 1 --2- 172.21.15.103:0/292460876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5b9910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009900 tx=0x7f1a88005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cade1072469efae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:06.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.610+0000 7f1a85c00640 1 -- 172.21.15.103:0/292460876 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88035040 con 0x557d1d5af590 2024-09-16T02:07:06.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.610+0000 7f1a85c00640 1 -- 172.21.15.103:0/292460876 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a88004760 con 0x557d1d5af590 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.610+0000 7f1a85c00640 1 -- 172.21.15.103:0/292460876 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88003970 con 0x557d1d5af590 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a9de9e740 1 -- 172.21.15.103:0/292460876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 msgr2=0x557d1d5b9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/292460876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5b9910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009900 tx=0x7f1a88005de0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a91600640 1 -- 172.21.15.103:0/4288667138 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1a800037a0 con 0x7f1a941049c0 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a9de9e740 1 -- 172.21.15.103:0/292460876 shutdown_connections 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/292460876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5b9e50 0x557d1d5bc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/292460876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5b9910 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/292460876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5aec70 0x557d1d5af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a9de9e740 1 -- 172.21.15.103:0/292460876 >> 172.21.15.103:0/292460876 conn(0x557d1d5ab200 msgr2=0x557d1d5ab5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a9de9e740 1 -- 172.21.15.103:0/292460876 shutdown_connections 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a9de9e740 1 -- 172.21.15.103:0/292460876 wait complete. 2024-09-16T02:07:06.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.611+0000 7f1a9de9e740 1 -- start start 2024-09-16T02:07:06.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a9de9e740 1 Processor -- start 2024-09-16T02:07:06.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5af590 0x557d1d5b7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d5b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d1d5bca20 con 0x557d1d5af590 2024-09-16T02:07:06.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557d1d4d00f0 con 0x557d1d5b9e50 2024-09-16T02:07:06.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557d1d5bcba0 con 0x557d1d5aec70 2024-09-16T02:07:06.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a91600640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1a7c0a31d0 0x7f1a7c0a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a9de9e740 10 client.0 _ll_register_callbacks cb 0x557d1d4b7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a91600640 1 -- 172.21.15.103:0/4288667138 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f1a800cfc20 con 0x7f1a941049c0 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.612+0000 7f1a99400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5af590 0x557d1d5b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a99400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:52176/0 (socket says 172.21.15.103:52176) 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a99400640 1 -- 172.21.15.103:0/4044502939 learned_addr learned my addr 172.21.15.103:0/4044502939 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a93e00640 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d5b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a99400640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1a7c0a31d0 0x7f1a7c0a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a99400640 1 -- 172.21.15.103:0/4044502939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 msgr2=0x557d1d5b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a99400640 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d5b78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a99400640 1 -- 172.21.15.103:0/4044502939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5af590 msgr2=0x557d1d5b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a99400640 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5af590 0x557d1d5b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a99400640 1 -- 172.21.15.103:0/4044502939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1d2863c0 con 0x557d1d5aec70 2024-09-16T02:07:06.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.613+0000 7f1a99400640 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b6e40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00f4f0 tx=0x7f1a8c00f9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19224d7fd193426f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:06.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.614+0000 7f1a7a000640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:07:06.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.614+0000 7f1a7a000640 1 -- 172.21.15.103:0/4044502939 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c022070 con 0x557d1d5aec70 2024-09-16T02:07:06.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.614+0000 7f1a7a000640 1 -- 172.21.15.103:0/4044502939 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f1a8c01e6d0 con 0x557d1d5aec70 2024-09-16T02:07:06.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.614+0000 7f1a99400640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1a7c0a31d0 0x7f1a7c0a5690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00feb0 tx=0x7f1a8c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:06.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.614+0000 7f1a9de9e740 1 -- 172.21.15.103:0/4044502939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a88004760 con 0x557d1d5aec70 2024-09-16T02:07:06.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.614+0000 7f1a7a000640 1 -- 172.21.15.103:0/4044502939 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1a8c01e9e0 con 0x557d1d5aec70 2024-09-16T02:07:06.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.614+0000 7f1a9de9e740 10 client.25339 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:06.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.614+0000 7f1a9de9e740 1 -- 172.21.15.103:0/4044502939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557d1d5c8be0 con 0x557d1d5aec70 2024-09-16T02:07:06.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.614+0000 7f1a7a000640 1 -- 172.21.15.103:0/4044502939 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a88004760 con 0x557d1d5aec70 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.615+0000 7f1a7a000640 1 -- 172.21.15.103:0/4044502939 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f1a8c027c20 con 0x557d1d5aec70 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.615+0000 7f1a9de9e740 10 client.25339 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.615+0000 7f1a9de9e740 10 client.25339 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.615+0000 7f1a9de9e740 20 client.25339 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.615+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x557d1d5ca140 0x557d1d5cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.615+0000 7f1a9de9e740 4 client.25339 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.615+0000 7f1a9de9e740 1 -- 172.21.15.103:0/4044502939 --> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d1d522950 con 0x557d1d5ca140 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.615+0000 7f1a98a00640 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x557d1d5ca140 0x557d1d5cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.616+0000 7f1a98a00640 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x557d1d5ca140 0x557d1d5cc5b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=546be7e59edcd608 server_cookie=baad47d021d94072 in_seq=0 out_seq=0 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.616+0000 7f1a7a000640 10 client.25339 ms_handle_connect on v2:172.21.15.136:6836/236571207 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.616+0000 7f1a7a000640 1 -- 172.21.15.103:0/4044502939 <== mds.1 v2:172.21.15.136:6836/236571207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d1d522950 con 0x557d1d5ca140 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.616+0000 7f1a7a000640 10 client.25339 handle_command_reply: tid=0 2024-09-16T02:07:06.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 client.25339 shutdown 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 -- 172.21.15.103:0/4044502939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 msgr2=0x557d1d5b6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b6e40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00f4f0 tx=0x7f1a8c00f9b0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 -- 172.21.15.103:0/4044502939 shutdown_connections 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x557d1d5ca140 0x557d1d5cc5b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d5b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5af590 0x557d1d5b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a7a000640 0 client.25339 ms_handle_reset on v2:172.21.15.136:6836/236571207 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/4044502939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b6e40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 -- 172.21.15.103:0/4044502939 >> 172.21.15.103:0/4044502939 conn(0x557d1d5ab200 msgr2=0x557d1d5b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 -- 172.21.15.103:0/4044502939 shutdown_connections 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 1 -- 172.21.15.103:0/4044502939 wait complete. 2024-09-16T02:07:06.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.617+0000 7f1a9de9e740 20 client.25339 trim_cache size 0 max 16384 2024-09-16T02:07:06.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.631+0000 7f1a9de9e740 1 Processor -- start 2024-09-16T02:07:06.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.631+0000 7f1a9de9e740 1 -- start start 2024-09-16T02:07:06.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5af590 0x557d1d5d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d5d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941b2230 con 0x557d1d5af590 2024-09-16T02:07:06.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941afb00 con 0x557d1d5b9e50 2024-09-16T02:07:06.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941ad950 con 0x557d1d5aec70 2024-09-16T02:07:06.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a99400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d5d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:52180/0 (socket says 172.21.15.103:52180) 2024-09-16T02:07:06.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5af590 0x557d1d5d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a99400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d5d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:47704/0 (socket says 172.21.15.103:47704) 2024-09-16T02:07:06.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.632+0000 7f1a98a00640 1 -- 172.21.15.103:0/2689011731 learned_addr learned my addr 172.21.15.103:0/2689011731 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:06.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.633+0000 7f1a98a00640 1 -- 172.21.15.103:0/2689011731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 msgr2=0x557d1d5d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.633+0000 7f1a98a00640 1 --2- 172.21.15.103:0/2689011731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d5d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.633+0000 7f1a98a00640 1 -- 172.21.15.103:0/2689011731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5af590 msgr2=0x557d1d5d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.633+0000 7f1a98a00640 1 --2- 172.21.15.103:0/2689011731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5af590 0x557d1d5d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.633+0000 7f1a98a00640 1 -- 172.21.15.103:0/2689011731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1d5a6770 con 0x557d1d5aec70 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.633+0000 7f1a98a00640 1 --2- 172.21.15.103:0/2689011731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b7510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1a88004f30 tx=0x7f1a88005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb16feded92db82 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.633+0000 7f1a85200640 1 -- 172.21.15.103:0/2689011731 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88035070 con 0x557d1d5aec70 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.633+0000 7f1a85200640 1 -- 172.21.15.103:0/2689011731 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a880054b0 con 0x557d1d5aec70 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.633+0000 7f1a85200640 1 -- 172.21.15.103:0/2689011731 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88003dc0 con 0x557d1d5aec70 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.634+0000 7f1a9de9e740 1 -- 172.21.15.103:0/2689011731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 msgr2=0x557d1d5b7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.634+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/2689011731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b7510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1a88004f30 tx=0x7f1a88005fd0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.634+0000 7f1a9de9e740 1 -- 172.21.15.103:0/2689011731 shutdown_connections 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.634+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/2689011731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d5d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.634+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/2689011731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5af590 0x557d1d5d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.634+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/2689011731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5aec70 0x557d1d5b7510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.634+0000 7f1a9de9e740 1 -- 172.21.15.103:0/2689011731 >> 172.21.15.103:0/2689011731 conn(0x557d1d5ab200 msgr2=0x557d1d5b6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:06.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.634+0000 7f1a9de9e740 1 -- 172.21.15.103:0/2689011731 shutdown_connections 2024-09-16T02:07:06.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.634+0000 7f1a9de9e740 1 -- 172.21.15.103:0/2689011731 wait complete. 2024-09-16T02:07:06.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.635+0000 7f1a9de9e740 1 -- start start 2024-09-16T02:07:06.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.635+0000 7f1a9de9e740 1 Processor -- start 2024-09-16T02:07:06.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.635+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5aec70 0x557d1d5bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.635+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5be4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.635+0000 7f1a9de9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d633e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.635+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941adfc0 con 0x557d1d5aec70 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.635+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941af430 con 0x557d1d5b9e50 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.635+0000 7f1a9de9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1a941ae690 con 0x557d1d5af590 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.635+0000 7f1a9de9e740 10 client.0 _ll_register_callbacks cb 0x7f1a8c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.635+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5aec70 0x557d1d5bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5be4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5be4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:52196/0 (socket says 172.21.15.103:52196) 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a93e00640 1 -- 172.21.15.103:0/3641022896 learned_addr learned my addr 172.21.15.103:0/3641022896 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a99400640 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d633e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5aec70 0x557d1d5bdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52854/0 (socket says 172.21.15.103:52854) 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a93e00640 1 -- 172.21.15.103:0/3641022896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 msgr2=0x557d1d633e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a93e00640 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d633e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a93e00640 1 -- 172.21.15.103:0/3641022896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5aec70 msgr2=0x557d1d5bdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a93e00640 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5aec70 0x557d1d5bdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a93e00640 1 -- 172.21.15.103:0/3641022896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1d634390 con 0x557d1d5af590 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a93e00640 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5be4e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a800cd640 tx=0x7f1a8009bb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb60f97569b56fc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a7a000640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a7a000640 1 -- 172.21.15.103:0/3641022896 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a800a0510 con 0x557d1d5af590 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a7a000640 1 -- 172.21.15.103:0/3641022896 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f1a800987d0 con 0x557d1d5af590 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.636+0000 7f1a9de9e740 1 -- 172.21.15.103:0/3641022896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a880054b0 con 0x557d1d5af590 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a7a000640 1 -- 172.21.15.103:0/3641022896 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1a800a4dd0 con 0x557d1d5af590 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a9de9e740 10 client.25345 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a9de9e740 1 -- 172.21.15.103:0/3641022896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557d1d6355b0 con 0x557d1d5af590 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a7a000640 1 -- 172.21.15.103:0/3641022896 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a880054b0 con 0x557d1d5af590 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a7a000640 1 -- 172.21.15.103:0/3641022896 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f1a8009b5b0 con 0x557d1d5af590 2024-09-16T02:07:06.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a9de9e740 10 client.25345 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a9de9e740 10 client.25345 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a9de9e740 20 client.25345 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x557d1d635d10 0x557d1d638130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a9de9e740 4 client.25345 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.637+0000 7f1a9de9e740 1 -- 172.21.15.103:0/3641022896 --> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d1d51ce60 con 0x557d1d635d10 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.638+0000 7f1a98a00640 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x557d1d635d10 0x557d1d638130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.638+0000 7f1a98a00640 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x557d1d635d10 0x557d1d638130 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d3af02afbf1c9d7 server_cookie=ff19fecb0ae9a122 in_seq=0 out_seq=0 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.638+0000 7f1a7a000640 10 client.25345 ms_handle_connect on v2:172.21.15.136:6836/236571207 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a7a000640 1 -- 172.21.15.103:0/3641022896 <== mds.1 v2:172.21.15.136:6836/236571207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d1d51ce60 con 0x557d1d635d10 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a7a000640 10 client.25345 handle_command_reply: tid=0 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 client.25345 shutdown 2024-09-16T02:07:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 -- 172.21.15.103:0/3641022896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 msgr2=0x557d1d5be4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5be4e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a800cd640 tx=0x7f1a8009bb80 comp rx=0 tx=0).stop 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 -- 172.21.15.103:0/3641022896 shutdown_connections 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x557d1d635d10 0x557d1d638130 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d1d5b9e50 0x557d1d633e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d1d5af590 0x557d1d5be4e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 --2- 172.21.15.103:0/3641022896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1d5aec70 0x557d1d5bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 -- 172.21.15.103:0/3641022896 >> 172.21.15.103:0/3641022896 conn(0x557d1d5ab200 msgr2=0x557d1d5b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a7a000640 0 client.25345 ms_handle_reset on v2:172.21.15.136:6836/236571207 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 -- 172.21.15.103:0/3641022896 shutdown_connections 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 1 -- 172.21.15.103:0/3641022896 wait complete. 2024-09-16T02:07:06.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.639+0000 7f1a9de9e740 20 client.25345 trim_cache size 0 max 16384 2024-09-16T02:07:06.646 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-16T02:07:06.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 -- 172.21.15.103:0/4288667138 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1a7c0a31d0 msgr2=0x7f1a7c0a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:06.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1a7c0a31d0 0x7f1a7c0a5690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00feb0 tx=0x7f1a8c01f000 comp rx=0 tx=0).stop 2024-09-16T02:07:06.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 -- 172.21.15.103:0/4288667138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a941049c0 msgr2=0x7f1a9419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:06.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a941049c0 0x7f1a9419ab80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000ea60 tx=0x7f1a8000ef20 comp rx=0 tx=0).stop 2024-09-16T02:07:06.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 -- 172.21.15.103:0/4288667138 shutdown_connections 2024-09-16T02:07:06.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1a7c0a31d0 0x7f1a7c0a5690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a94109140 0x7f1a9419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a941049c0 0x7f1a9419ab80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 --2- 172.21.15.103:0/4288667138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a94104010 0x7f1a9419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:06.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 -- 172.21.15.103:0/4288667138 >> 172.21.15.103:0/4288667138 conn(0x7f1a940ffa50 msgr2=0x7f1a94100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:06.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.640+0000 7f1a9b400640 1 -- 172.21.15.103:0/4288667138 shutdown_connections 2024-09-16T02:07:06.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:06.641+0000 7f1a9b400640 1 -- 172.21.15.103:0/4288667138 wait complete. 2024-09-16T02:07:06.742 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-16T02:07:06.742 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-09-16T02:07:06.742 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-16T02:07:06.742 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-09-16T02:07:06.742 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-16T02:07:06.742 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-09-16T02:07:06.742 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-16T02:07:06.743 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-16T02:07:06.744 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-16T02:07:06.745 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-16T02:07:06.746 INFO:tasks.workunit.client.0.smithi103.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-16T02:07:06.747 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-09-16T02:07:06.748 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-16T02:07:06.748 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-09-16T02:07:06.748 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-16T02:07:06.748 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-09-16T02:07:06.748 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-16T02:07:06.748 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-09-16T02:07:06.748 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-16T02:07:06.748 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-09-16T02:07:07.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-16T02:07:07.530 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-16T02:07:07.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-16T02:07:07.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-09-16T02:07:07.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-09-16T02:07:07.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-16T02:07:07.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-09-16T02:07:07.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-16T02:07:07.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-09-16T02:07:07.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-16T02:07:07.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-09-16T02:07:07.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-16T02:07:07.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-09-16T02:07:07.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-16T02:07:07.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-09-16T02:07:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:07.981 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:07 smithi103 ceph-mon[30398]: pgmap v390: 97 pgs: 97 active+clean; 1.2 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 KiB/s wr, 1.75k op/s 2024-09-16T02:07:08.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:07 smithi136 ceph-mon[35323]: pgmap v390: 97 pgs: 97 active+clean; 1.2 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 KiB/s wr, 1.75k op/s 2024-09-16T02:07:08.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:07 smithi172 ceph-mon[35340]: pgmap v390: 97 pgs: 97 active+clean; 1.2 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 KiB/s wr, 1.75k op/s 2024-09-16T02:07:08.088 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-16T02:07:08.088 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-09-16T02:07:08.088 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-16T02:07:08.088 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-09-16T02:07:08.088 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-16T02:07:08.088 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-09-16T02:07:08.089 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-16T02:07:08.090 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-16T02:07:08.091 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-16T02:07:08.092 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-09-16T02:07:08.093 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-09-16T02:07:08.094 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-16T02:07:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.392+0000 7f9ec9a00640 1 -- 172.21.15.103:0/1597936876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec41019c0 msgr2=0x7f9ec4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.392+0000 7f9ec9a00640 1 --2- 172.21.15.103:0/1597936876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec41019c0 0x7f9ec4101e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009930 tx=0x7f9eb002a420 comp rx=0 tx=0).stop 2024-09-16T02:07:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.392+0000 7f9ec9a00640 1 -- 172.21.15.103:0/1597936876 shutdown_connections 2024-09-16T02:07:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.392+0000 7f9ec9a00640 1 --2- 172.21.15.103:0/1597936876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec4102360 0x7f9ec410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.392+0000 7f9ec9a00640 1 --2- 172.21.15.103:0/1597936876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec41019c0 0x7f9ec4101e20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.392+0000 7f9ec9a00640 1 --2- 172.21.15.103:0/1597936876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec41079c0 0x7f9ec4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.392+0000 7f9ec9a00640 1 -- 172.21.15.103:0/1597936876 >> 172.21.15.103:0/1597936876 conn(0x7f9ec40fd740 msgr2=0x7f9ec40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:08.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.392+0000 7f9ec9a00640 1 -- 172.21.15.103:0/1597936876 shutdown_connections 2024-09-16T02:07:08.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.392+0000 7f9ec9a00640 1 -- 172.21.15.103:0/1597936876 wait complete. 2024-09-16T02:07:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.393+0000 7f9ec9a00640 1 Processor -- start 2024-09-16T02:07:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.393+0000 7f9ec9a00640 1 -- start start 2024-09-16T02:07:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.393+0000 7f9ec9a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec41019c0 0x7f9ec40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.393+0000 7f9ec9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec4102360 0x7f9ec4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.393+0000 7f9ec9a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec41079c0 0x7f9ec4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.393+0000 7f9ec9a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4111e10 con 0x7f9ec41079c0 2024-09-16T02:07:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.393+0000 7f9ec9a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4111c90 con 0x7f9ec41019c0 2024-09-16T02:07:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.393+0000 7f9ec9a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4111f90 con 0x7f9ec4102360 2024-09-16T02:07:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec41019c0 0x7f9ec40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec41019c0 0x7f9ec40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49028/0 (socket says 172.21.15.103:49028) 2024-09-16T02:07:08.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec4102360 0x7f9ec4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec3e00640 1 -- 172.21.15.103:0/3229113022 learned_addr learned my addr 172.21.15.103:0/3229113022 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:08.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec9000640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec41079c0 0x7f9ec4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec3e00640 1 -- 172.21.15.103:0/3229113022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec4102360 msgr2=0x7f9ec4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:08.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec4102360 0x7f9ec4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec3e00640 1 -- 172.21.15.103:0/3229113022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec41079c0 msgr2=0x7f9ec4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:08.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec41079c0 0x7f9ec4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec3e00640 1 -- 172.21.15.103:0/3229113022 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec41a6780 con 0x7f9ec41019c0 2024-09-16T02:07:08.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.394+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec41019c0 0x7f9ec40744f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400e9f0 tx=0x7f9eb400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:08.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.395+0000 7f9ec0c00640 1 -- 172.21.15.103:0/3229113022 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400cd80 con 0x7f9ec41019c0 2024-09-16T02:07:08.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.395+0000 7f9ec0c00640 1 -- 172.21.15.103:0/3229113022 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4005500 con 0x7f9ec41019c0 2024-09-16T02:07:08.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.395+0000 7f9ec0c00640 1 -- 172.21.15.103:0/3229113022 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4010690 con 0x7f9ec41019c0 2024-09-16T02:07:08.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.395+0000 7f9ec9a00640 1 -- 172.21.15.103:0/3229113022 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec41a6a70 con 0x7f9ec41019c0 2024-09-16T02:07:08.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.395+0000 7f9ec9a00640 1 -- 172.21.15.103:0/3229113022 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec41a6f30 con 0x7f9ec41019c0 2024-09-16T02:07:08.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.398+0000 7f9ecdc29740 1 Processor -- start 2024-09-16T02:07:08.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.398+0000 7f9ec0c00640 1 -- 172.21.15.103:0/3229113022 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9eb40046c0 con 0x7f9ec41019c0 2024-09-16T02:07:08.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.398+0000 7f9ecdc29740 1 -- start start 2024-09-16T02:07:08.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.399+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 0x556af5e7b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.399+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e7b760 0x556af5e85ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.399+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e86020 0x556af5e88410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.399+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556af5de13e0 con 0x556af5e7b760 2024-09-16T02:07:08.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.399+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556af5d88d80 con 0x556af5e7ae40 2024-09-16T02:07:08.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.399+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556af5ddd3a0 con 0x556af5e86020 2024-09-16T02:07:08.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.399+0000 7f9ec9000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 0x556af5e7b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.399+0000 7f9ec9000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 0x556af5e7b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49040/0 (socket says 172.21.15.103:49040) 2024-09-16T02:07:08.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.399+0000 7f9ec9000640 1 -- 172.21.15.103:0/2819029743 learned_addr learned my addr 172.21.15.103:0/2819029743 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:08.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.399+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/2819029743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e7b760 0x556af5e85ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9ec9000640 1 -- 172.21.15.103:0/2819029743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e86020 msgr2=0x556af5e88410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9ec9000640 1 --2- 172.21.15.103:0/2819029743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e86020 0x556af5e88410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9ec9000640 1 -- 172.21.15.103:0/2819029743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e7b760 msgr2=0x556af5e85ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9ec9000640 1 --2- 172.21.15.103:0/2819029743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e7b760 0x556af5e85ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9ec9000640 1 -- 172.21.15.103:0/2819029743 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556af5da7ae0 con 0x556af5e7ae40 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9ec9000640 1 --2- 172.21.15.103:0/2819029743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 0x556af5e7b220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009fd0 tx=0x7f9ebc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da60bdab21090691 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9ec0c00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3240 0x7f9ea80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9eb9600640 1 -- 172.21.15.103:0/2819029743 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc004d20 con 0x556af5e7ae40 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9eb9600640 1 -- 172.21.15.103:0/2819029743 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ebc004050 con 0x556af5e7ae40 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9eb9600640 1 -- 172.21.15.103:0/2819029743 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc005280 con 0x556af5e7ae40 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3240 0x7f9ea80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.400+0000 7f9ec0c00640 1 -- 172.21.15.103:0/3229113022 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f9eb40cfda0 con 0x7f9ec41019c0 2024-09-16T02:07:08.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.401+0000 7f9ecdc29740 1 -- 172.21.15.103:0/2819029743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 msgr2=0x556af5e7b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:08.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.401+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/2819029743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 0x556af5e7b220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009fd0 tx=0x7f9ebc0118d0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.401+0000 7f9ecdc29740 1 -- 172.21.15.103:0/2819029743 shutdown_connections 2024-09-16T02:07:08.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.401+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/2819029743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e86020 0x556af5e88410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.401+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/2819029743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e7b760 0x556af5e85ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.401+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/2819029743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 0x556af5e7b220 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.401+0000 7f9ecdc29740 1 -- 172.21.15.103:0/2819029743 >> 172.21.15.103:0/2819029743 conn(0x556af5e773d0 msgr2=0x556af5e777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.401+0000 7f9ec3e00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3240 0x7f9ea80a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4098740 tx=0x7f9eb4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.401+0000 7f9ecdc29740 1 -- 172.21.15.103:0/2819029743 shutdown_connections 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.401+0000 7f9ecdc29740 1 -- 172.21.15.103:0/2819029743 wait complete. 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.402+0000 7f9ecdc29740 1 -- start start 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.402+0000 7f9ecdc29740 1 Processor -- start 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.402+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e7ae40 0x556af5e82f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.402+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e83460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.402+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e86020 0x556af5e839a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.402+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556af5e88bf0 con 0x556af5e7ae40 2024-09-16T02:07:08.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.402+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556af5d9c0f0 con 0x556af5e86020 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.402+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556af5e88d70 con 0x556af5e7b760 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ecdc29740 10 client.0 _ll_register_callbacks cb 0x556af5d83bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e83460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e83460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:37862/0 (socket says 172.21.15.103:37862) 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec3400640 1 -- 172.21.15.103:0/4182566846 learned_addr learned my addr 172.21.15.103:0/4182566846 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec9000640 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e86020 0x556af5e839a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec9000640 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e7ae40 0x556af5e82f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec3400640 1 -- 172.21.15.103:0/4182566846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e86020 msgr2=0x556af5e839a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec3400640 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e86020 0x556af5e839a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec3400640 1 -- 172.21.15.103:0/4182566846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e7ae40 msgr2=0x556af5e82f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec3400640 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e7ae40 0x556af5e82f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec3400640 1 -- 172.21.15.103:0/4182566846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af5b523c0 con 0x556af5e7b760 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9ec3400640 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e83460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0033000 tx=0x7f9eb0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b034bc78ebb327e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.403+0000 7f9eae000640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.404+0000 7f9eae000640 1 -- 172.21.15.103:0/4182566846 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0040070 con 0x556af5e7b760 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.404+0000 7f9eae000640 1 -- 172.21.15.103:0/4182566846 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f9eb0005250 con 0x556af5e7b760 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.404+0000 7f9ecdc29740 1 -- 172.21.15.103:0/4182566846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ebc004050 con 0x556af5e7b760 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.404+0000 7f9eae000640 1 -- 172.21.15.103:0/4182566846 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9eb004c050 con 0x556af5e7b760 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.404+0000 7f9ecdc29740 10 client.25354 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.404+0000 7f9ecdc29740 1 -- 172.21.15.103:0/4182566846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556af5e94c40 con 0x556af5e7b760 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.404+0000 7f9eae000640 1 -- 172.21.15.103:0/4182566846 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ebc004050 con 0x556af5e7b760 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.405+0000 7f9eae000640 1 -- 172.21.15.103:0/4182566846 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9eb00037b0 con 0x556af5e7b760 2024-09-16T02:07:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.405+0000 7f9ecdc29740 10 client.25354 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.405+0000 7f9ecdc29740 10 client.25354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.405+0000 7f9ecdc29740 20 client.25354 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.405+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556af5e96050 0x556af5e98480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.405+0000 7f9ecdc29740 4 client.25354 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.405+0000 7f9ecdc29740 1 -- 172.21.15.103:0/4182566846 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556af5dee950 con 0x556af5e96050 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.405+0000 7f9ec9000640 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556af5e96050 0x556af5e98480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.405+0000 7f9ec9000640 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556af5e96050 0x556af5e98480 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1da35ff12b7dc55 server_cookie=433e49c8cfc43907 in_seq=0 out_seq=0 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.405+0000 7f9eae000640 10 client.25354 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9eae000640 1 -- 172.21.15.103:0/4182566846 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556af5dee950 con 0x556af5e96050 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9eae000640 10 client.25354 handle_command_reply: tid=0 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9ecdc29740 1 client.25354 shutdown 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9ecdc29740 1 -- 172.21.15.103:0/4182566846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 msgr2=0x556af5e83460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e83460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0033000 tx=0x7f9eb0004270 comp rx=0 tx=0).stop 2024-09-16T02:07:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9ecdc29740 1 -- 172.21.15.103:0/4182566846 shutdown_connections 2024-09-16T02:07:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556af5e96050 0x556af5e98480 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e86020 0x556af5e839a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e83460 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9eae000640 0 client.25354 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/4182566846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e7ae40 0x556af5e82f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.407+0000 7f9ecdc29740 1 -- 172.21.15.103:0/4182566846 >> 172.21.15.103:0/4182566846 conn(0x556af5e773d0 msgr2=0x556af5e7d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.408+0000 7f9ecdc29740 1 -- 172.21.15.103:0/4182566846 shutdown_connections 2024-09-16T02:07:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.408+0000 7f9ecdc29740 1 -- 172.21.15.103:0/4182566846 wait complete. 2024-09-16T02:07:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.408+0000 7f9ecdc29740 20 client.25354 trim_cache size 0 max 16384 2024-09-16T02:07:08.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.422+0000 7f9ecdc29740 1 Processor -- start 2024-09-16T02:07:08.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ecdc29740 1 -- start start 2024-09-16T02:07:08.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 0x556af5e7d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e9dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 0x556af5e9e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec41b2150 con 0x556af5e86020 2024-09-16T02:07:08.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec41afa20 con 0x556af5e7ae40 2024-09-16T02:07:08.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec41ad870 con 0x556af5e7b760 2024-09-16T02:07:08.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ec3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e9dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ec3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e9dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:37872/0 (socket says 172.21.15.103:37872) 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ec3400640 1 -- 172.21.15.103:0/2639797164 learned_addr learned my addr 172.21.15.103:0/2639797164 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ec9000640 1 --2- 172.21.15.103:0/2639797164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 0x556af5e7d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.423+0000 7f9ec9000640 1 --2- 172.21.15.103:0/2639797164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 0x556af5e9e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.424+0000 7f9ec3400640 1 -- 172.21.15.103:0/2639797164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 msgr2=0x556af5e7d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.424+0000 7f9ec3400640 1 --2- 172.21.15.103:0/2639797164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 0x556af5e7d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.424+0000 7f9ec3400640 1 -- 172.21.15.103:0/2639797164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 msgr2=0x556af5e9e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.424+0000 7f9ec3400640 1 --2- 172.21.15.103:0/2639797164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 0x556af5e9e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.424+0000 7f9ec3400640 1 -- 172.21.15.103:0/2639797164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556af5e85890 con 0x556af5e7b760 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.424+0000 7f9ec9000640 1 --2- 172.21.15.103:0/2639797164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 0x556af5e9e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:07:08.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.424+0000 7f9ec3400640 1 --2- 172.21.15.103:0/2639797164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e9dd50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0031550 tx=0x7f9eb0009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f38ac86e6ac4be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:08.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.424+0000 7f9eb8c00640 1 -- 172.21.15.103:0/2639797164 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0040040 con 0x556af5e7b760 2024-09-16T02:07:08.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.424+0000 7f9eb8c00640 1 -- 172.21.15.103:0/2639797164 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb0009780 con 0x556af5e7b760 2024-09-16T02:07:08.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.424+0000 7f9eb8c00640 1 -- 172.21.15.103:0/2639797164 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0035040 con 0x556af5e7b760 2024-09-16T02:07:08.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.425+0000 7f9ecdc29740 1 -- 172.21.15.103:0/2639797164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 msgr2=0x556af5e9dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:08.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.425+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/2639797164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e9dd50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0031550 tx=0x7f9eb0009230 comp rx=0 tx=0).stop 2024-09-16T02:07:08.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.425+0000 7f9ecdc29740 1 -- 172.21.15.103:0/2639797164 shutdown_connections 2024-09-16T02:07:08.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.425+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/2639797164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 0x556af5e9e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.425+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/2639797164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7b760 0x556af5e9dd50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.425+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/2639797164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7ae40 0x556af5e7d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.425+0000 7f9ecdc29740 1 -- 172.21.15.103:0/2639797164 >> 172.21.15.103:0/2639797164 conn(0x556af5e773d0 msgr2=0x556af5e84fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:08.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.425+0000 7f9ecdc29740 1 -- 172.21.15.103:0/2639797164 shutdown_connections 2024-09-16T02:07:08.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.425+0000 7f9ecdc29740 1 -- 172.21.15.103:0/2639797164 wait complete. 2024-09-16T02:07:08.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ecdc29740 1 -- start start 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ecdc29740 1 Processor -- start 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7ae40 0x556af5e8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7b760 0x556af5e8e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ecdc29740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 0x556af5e8af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec41adee0 con 0x556af5e86020 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec41af350 con 0x556af5e7b760 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ecdc29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec41ae5b0 con 0x556af5e7ae40 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ecdc29740 10 client.0 _ll_register_callbacks cb 0x7f9eb4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ec3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7b760 0x556af5e8e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ec3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7b760 0x556af5e8e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49068/0 (socket says 172.21.15.103:49068) 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.426+0000 7f9ec3400640 1 -- 172.21.15.103:0/1312829082 learned_addr learned my addr 172.21.15.103:0/1312829082 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9ec9000640 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7ae40 0x556af5e8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9ec9000640 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 0x556af5e8af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9ec3400640 1 -- 172.21.15.103:0/1312829082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7ae40 msgr2=0x556af5e8dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9ec3400640 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7ae40 0x556af5e8dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9ec3400640 1 -- 172.21.15.103:0/1312829082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 msgr2=0x556af5e8af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9ec3400640 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 0x556af5e8af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9ec3400640 1 -- 172.21.15.103:0/1312829082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af5e8b4d0 con 0x556af5e7b760 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9ec9000640 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 0x556af5e8af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9ec3400640 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7b760 0x556af5e8e500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x556af5e85d60 tx=0x7f9eb0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fc218d2c0d0f149 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9eae000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9eae000640 1 -- 172.21.15.103:0/1312829082 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0035040 con 0x556af5e7b760 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.427+0000 7f9ecdc29740 1 -- 172.21.15.103:0/1312829082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eb0009780 con 0x556af5e7b760 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9eae000640 1 -- 172.21.15.103:0/1312829082 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f9eb0005450 con 0x556af5e7b760 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9eae000640 1 -- 172.21.15.103:0/1312829082 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9eb005d030 con 0x556af5e7b760 2024-09-16T02:07:08.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9ecdc29740 10 client.25388 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9ecdc29740 1 -- 172.21.15.103:0/1312829082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556af5f09af0 con 0x556af5e7b760 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9eae000640 1 -- 172.21.15.103:0/1312829082 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb0009780 con 0x556af5e7b760 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9eae000640 1 -- 172.21.15.103:0/1312829082 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9eb00318b0 con 0x556af5e7b760 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9ecdc29740 10 client.25388 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9ecdc29740 10 client.25388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9ecdc29740 20 client.25388 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556af5f0a200 0x556af5f0c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9ecdc29740 4 client.25388 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.428+0000 7f9ecdc29740 1 -- 172.21.15.103:0/1312829082 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556af5de8e60 con 0x556af5f0a200 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.429+0000 7f9ec9000640 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556af5f0a200 0x556af5f0c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.429+0000 7f9ec9000640 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556af5f0a200 0x556af5f0c620 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20c9ab312ce63418 server_cookie=3bf14fd84f9d9ede in_seq=0 out_seq=0 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.429+0000 7f9eae000640 10 client.25388 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:08.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.429+0000 7f9eae000640 1 -- 172.21.15.103:0/1312829082 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556af5de8e60 con 0x556af5f0a200 2024-09-16T02:07:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.429+0000 7f9eae000640 10 client.25388 handle_command_reply: tid=0 2024-09-16T02:07:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.429+0000 7f9ecdc29740 1 client.25388 shutdown 2024-09-16T02:07:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 1 -- 172.21.15.103:0/1312829082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7b760 msgr2=0x556af5e8e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7b760 0x556af5e8e500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x556af5e85d60 tx=0x7f9eb0005850 comp rx=0 tx=0).stop 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 1 -- 172.21.15.103:0/1312829082 shutdown_connections 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x556af5f0a200 0x556af5f0c620 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556af5e86020 0x556af5e8af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556af5e7b760 0x556af5e8e500 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9eae000640 0 client.25388 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 1 --2- 172.21.15.103:0/1312829082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556af5e7ae40 0x556af5e8dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 1 -- 172.21.15.103:0/1312829082 >> 172.21.15.103:0/1312829082 conn(0x556af5e773d0 msgr2=0x556af5e84fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 1 -- 172.21.15.103:0/1312829082 shutdown_connections 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 1 -- 172.21.15.103:0/1312829082 wait complete. 2024-09-16T02:07:08.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.430+0000 7f9ecdc29740 20 client.25388 trim_cache size 0 max 16384 2024-09-16T02:07:08.438 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-16T02:07:08.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 -- 172.21.15.103:0/3229113022 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3240 msgr2=0x7f9ea80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:08.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3240 0x7f9ea80a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4098740 tx=0x7f9eb4004a40 comp rx=0 tx=0).stop 2024-09-16T02:07:08.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 -- 172.21.15.103:0/3229113022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec41019c0 msgr2=0x7f9ec40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:08.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec41019c0 0x7f9ec40744f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400e9f0 tx=0x7f9eb400edf0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 -- 172.21.15.103:0/3229113022 shutdown_connections 2024-09-16T02:07:08.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f9ea80a3240 0x7f9ea80a5700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec41079c0 0x7f9ec4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ec4102360 0x7f9ec4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 --2- 172.21.15.103:0/3229113022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ec41019c0 0x7f9ec40744f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:08.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 -- 172.21.15.103:0/3229113022 >> 172.21.15.103:0/3229113022 conn(0x7f9ec40fd740 msgr2=0x7f9ec40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:08.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 -- 172.21.15.103:0/3229113022 shutdown_connections 2024-09-16T02:07:08.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:08.431+0000 7f9ec9a00640 1 -- 172.21.15.103:0/3229113022 wait complete. 2024-09-16T02:07:08.487 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-09-16T02:07:08.487 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-16T02:07:08.487 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-09-16T02:07:08.487 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-16T02:07:08.487 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-09-16T02:07:08.488 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-16T02:07:08.489 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-09-16T02:07:08.490 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-16T02:07:08.490 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-09-16T02:07:08.490 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-16T02:07:08.490 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-09-16T02:07:08.490 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-16T02:07:08.490 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-09-16T02:07:08.490 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-16T02:07:08.490 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-09-16T02:07:08.490 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-16T02:07:08.490 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-09-16T02:07:08.491 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-16T02:07:08.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-09-16T02:07:08.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-16T02:07:08.494 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-09-16T02:07:08.494 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-16T02:07:08.494 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-09-16T02:07:08.494 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-16T02:07:08.494 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-09-16T02:07:08.494 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-16T02:07:09.194 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-09-16T02:07:09.195 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-09-16T02:07:09.196 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-09-16T02:07:09.197 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-09-16T02:07:09.198 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-09-16T02:07:09.199 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-09-16T02:07:09.200 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-16T02:07:09.965 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-09-16T02:07:09.965 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-16T02:07:09.965 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-09-16T02:07:09.965 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-16T02:07:09.965 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-09-16T02:07:09.965 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-16T02:07:09.965 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-09-16T02:07:09.965 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-16T02:07:09.965 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-09-16T02:07:09.965 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-16T02:07:09.966 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-16T02:07:09.967 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-16T02:07:09.968 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-16T02:07:09.969 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-16T02:07:09.970 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-09-16T02:07:09.971 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-16T02:07:09.971 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-09-16T02:07:09.971 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-16T02:07:10.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:09 smithi103 ceph-mon[30398]: pgmap v391: 97 pgs: 97 active+clean; 42 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 438 KiB/s wr, 1.86k op/s 2024-09-16T02:07:10.022 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:09 smithi136 ceph-mon[35323]: pgmap v391: 97 pgs: 97 active+clean; 42 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 438 KiB/s wr, 1.86k op/s 2024-09-16T02:07:10.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:09 smithi172 ceph-mon[35340]: pgmap v391: 97 pgs: 97 active+clean; 42 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 438 KiB/s wr, 1.86k op/s 2024-09-16T02:07:10.241 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T02:07:10.241 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T02:07:10.339 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-09-16T02:07:10.339 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-16T02:07:10.339 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-16T02:07:10.340 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-16T02:07:10.341 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-09-16T02:07:10.342 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-16T02:07:10.343 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-09-16T02:07:10.344 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-16T02:07:10.345 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-09-16T02:07:10.346 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-16T02:07:10.640 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:10.730 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-09-16T02:07:10.730 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-16T02:07:10.730 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-09-16T02:07:10.730 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-16T02:07:10.730 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-09-16T02:07:10.730 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-16T02:07:10.730 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-09-16T02:07:10.730 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-16T02:07:10.730 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-09-16T02:07:10.730 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-16T02:07:10.731 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-09-16T02:07:10.732 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-09-16T02:07:10.733 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-16T02:07:10.734 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-16T02:07:10.735 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-09-16T02:07:10.736 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-16T02:07:10.736 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-09-16T02:07:10.736 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-16T02:07:10.736 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-09-16T02:07:10.736 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-16T02:07:10.736 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-09-16T02:07:11.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:10 smithi103 ceph-mon[30398]: pgmap v392: 97 pgs: 97 active+clean; 162 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 820 KiB/s wr, 800 op/s 2024-09-16T02:07:11.022 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:10 smithi136 ceph-mon[35323]: pgmap v392: 97 pgs: 97 active+clean; 162 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 820 KiB/s wr, 800 op/s 2024-09-16T02:07:11.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:10 smithi172 ceph-mon[35340]: pgmap v392: 97 pgs: 97 active+clean; 162 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 820 KiB/s wr, 800 op/s 2024-09-16T02:07:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.260+0000 7f19eb800640 1 -- 172.21.15.103:0/3453106640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e40ffe60 msgr2=0x7f19e4100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.260+0000 7f19eb800640 1 --2- 172.21.15.103:0/3453106640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e40ffe60 0x7f19e4100240 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009900 tx=0x7f19d802a4a0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.260+0000 7f19eb800640 1 -- 172.21.15.103:0/3453106640 shutdown_connections 2024-09-16T02:07:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.260+0000 7f19eb800640 1 --2- 172.21.15.103:0/3453106640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e410a2f0 0x7f19e4110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.260+0000 7f19eb800640 1 --2- 172.21.15.103:0/3453106640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e4100780 0x7f19e4109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.260+0000 7f19eb800640 1 --2- 172.21.15.103:0/3453106640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e40ffe60 0x7f19e4100240 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.260+0000 7f19eb800640 1 -- 172.21.15.103:0/3453106640 >> 172.21.15.103:0/3453106640 conn(0x7f19e40fba90 msgr2=0x7f19e40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.261+0000 7f19eb800640 1 -- 172.21.15.103:0/3453106640 shutdown_connections 2024-09-16T02:07:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.261+0000 7f19eb800640 1 -- 172.21.15.103:0/3453106640 wait complete. 2024-09-16T02:07:11.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.263+0000 7f19eb800640 1 Processor -- start 2024-09-16T02:07:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.263+0000 7f19eb800640 1 -- start start 2024-09-16T02:07:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.264+0000 7f19eb800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e40ffe60 0x7f19e419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.264+0000 7f19eb800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e4100780 0x7f19e419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.264+0000 7f19eb800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e410a2f0 0x7f19e419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.264+0000 7f19eb800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4114200 con 0x7f19e4100780 2024-09-16T02:07:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.264+0000 7f19eb800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4114080 con 0x7f19e410a2f0 2024-09-16T02:07:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.264+0000 7f19eb800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4114380 con 0x7f19e40ffe60 2024-09-16T02:07:11.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.264+0000 7f19e8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e40ffe60 0x7f19e419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.264+0000 7f19e8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e40ffe60 0x7f19e419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:37882/0 (socket says 172.21.15.103:37882) 2024-09-16T02:07:11.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.264+0000 7f19e8e00640 1 -- 172.21.15.103:0/2043006428 learned_addr learned my addr 172.21.15.103:0/2043006428 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:11.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.264+0000 7f19e3e00640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e4100780 0x7f19e419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.265+0000 7f19e9800640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e410a2f0 0x7f19e419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.265+0000 7f19e9800640 1 -- 172.21.15.103:0/2043006428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e40ffe60 msgr2=0x7f19e419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:11.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.265+0000 7f19e9800640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e40ffe60 0x7f19e419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.265+0000 7f19e9800640 1 -- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e4100780 msgr2=0x7f19e419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:11.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.265+0000 7f19e9800640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e4100780 0x7f19e419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.265+0000 7f19e9800640 1 -- 172.21.15.103:0/2043006428 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e419f6d0 con 0x7f19e410a2f0 2024-09-16T02:07:11.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.265+0000 7f19e3e00640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e4100780 0x7f19e419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:11.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.266+0000 7f19e9800640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e410a2f0 0x7f19e419eff0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19dc013380 tx=0x7f19dc013840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:11.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.266+0000 7f19e1600640 1 -- 172.21.15.103:0/2043006428 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00fcd0 con 0x7f19e410a2f0 2024-09-16T02:07:11.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.266+0000 7f19e1600640 1 -- 172.21.15.103:0/2043006428 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19dc01f070 con 0x7f19e410a2f0 2024-09-16T02:07:11.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.266+0000 7f19e1600640 1 -- 172.21.15.103:0/2043006428 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc01a830 con 0x7f19e410a2f0 2024-09-16T02:07:11.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.266+0000 7f19eb800640 1 -- 172.21.15.103:0/2043006428 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e419f9c0 con 0x7f19e410a2f0 2024-09-16T02:07:11.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.267+0000 7f19eb800640 1 -- 172.21.15.103:0/2043006428 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e41037b0 con 0x7f19e410a2f0 2024-09-16T02:07:11.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.273+0000 7f19ee411740 1 Processor -- start 2024-09-16T02:07:11.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.274+0000 7f19e1600640 1 -- 172.21.15.103:0/2043006428 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f19dc01aa10 con 0x7f19e410a2f0 2024-09-16T02:07:11.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.274+0000 7f19ee411740 1 -- start start 2024-09-16T02:07:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.274+0000 7f19ee411740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 0x55c4a51ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.275+0000 7f19ee411740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 0x55c4a51e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.275+0000 7f19ee411740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a51eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.275+0000 7f19ee411740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5144290 con 0x55c4a51e8ca0 2024-09-16T02:07:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.275+0000 7f19ee411740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c4a50ebb40 con 0x55c4a51ddac0 2024-09-16T02:07:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.275+0000 7f19ee411740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5140250 con 0x55c4a51de3e0 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.275+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 0x55c4a51ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.275+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 0x55c4a51ddea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49084/0 (socket says 172.21.15.103:49084) 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.275+0000 7f19e3e00640 1 -- 172.21.15.103:0/3128747287 learned_addr learned my addr 172.21.15.103:0/3128747287 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.275+0000 7f19e3e00640 1 --2- 172.21.15.103:0/3128747287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a51eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.275+0000 7f19e8e00640 1 --2- 172.21.15.103:0/3128747287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 0x55c4a51e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.276+0000 7f19e3e00640 1 -- 172.21.15.103:0/3128747287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 msgr2=0x55c4a51e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.276+0000 7f19e3e00640 1 --2- 172.21.15.103:0/3128747287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 0x55c4a51e8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.276+0000 7f19e3e00640 1 -- 172.21.15.103:0/3128747287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 msgr2=0x55c4a51eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.276+0000 7f19e3e00640 1 --2- 172.21.15.103:0/3128747287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a51eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.276+0000 7f19e3e00640 1 -- 172.21.15.103:0/3128747287 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a4eb56a0 con 0x55c4a51ddac0 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.276+0000 7f19e1600640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f19cc0a3300 0x7f19cc0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.276+0000 7f19e3e00640 1 --2- 172.21.15.103:0/3128747287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 0x55c4a51ddea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f19d000bfd0 tx=0x7f19d000bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e16aa162396b85af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:11.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.277+0000 7f19e1600640 1 -- 172.21.15.103:0/2043006428 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f19dc0d7ad0 con 0x7f19e410a2f0 2024-09-16T02:07:11.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.276+0000 7f19d6000640 1 -- 172.21.15.103:0/3128747287 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0004020 con 0x55c4a51ddac0 2024-09-16T02:07:11.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.277+0000 7f19d6000640 1 -- 172.21.15.103:0/3128747287 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19d0010d10 con 0x55c4a51ddac0 2024-09-16T02:07:11.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.277+0000 7f19d6000640 1 -- 172.21.15.103:0/3128747287 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0003a80 con 0x55c4a51ddac0 2024-09-16T02:07:11.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.277+0000 7f19e8e00640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f19cc0a3300 0x7f19cc0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.278+0000 7f19ee411740 1 -- 172.21.15.103:0/3128747287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 msgr2=0x55c4a51ddea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.278+0000 7f19ee411740 1 --2- 172.21.15.103:0/3128747287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 0x55c4a51ddea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f19d000bfd0 tx=0x7f19d000bb70 comp rx=0 tx=0).stop 2024-09-16T02:07:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.278+0000 7f19e8e00640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f19cc0a3300 0x7f19cc0a57c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f19d8005c40 tx=0x7f19d802a9a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.279+0000 7f19ee411740 1 -- 172.21.15.103:0/3128747287 shutdown_connections 2024-09-16T02:07:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.279+0000 7f19ee411740 1 --2- 172.21.15.103:0/3128747287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a51eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.279+0000 7f19ee411740 1 --2- 172.21.15.103:0/3128747287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 0x55c4a51e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.279+0000 7f19ee411740 1 --2- 172.21.15.103:0/3128747287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 0x55c4a51ddea0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.279+0000 7f19ee411740 1 -- 172.21.15.103:0/3128747287 >> 172.21.15.103:0/3128747287 conn(0x55c4a51da050 msgr2=0x55c4a51da430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.279+0000 7f19ee411740 1 -- 172.21.15.103:0/3128747287 shutdown_connections 2024-09-16T02:07:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.280+0000 7f19ee411740 1 -- 172.21.15.103:0/3128747287 wait complete. 2024-09-16T02:07:11.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.280+0000 7f19ee411740 1 -- start start 2024-09-16T02:07:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.282+0000 7f19ee411740 1 Processor -- start 2024-09-16T02:07:11.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.283+0000 7f19ee411740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51ddac0 0x55c4a51e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.284+0000 7f19ee411740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.284+0000 7f19ee411740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a51e5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.284+0000 7f19ee411740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4a51eb910 con 0x55c4a51e8ca0 2024-09-16T02:07:11.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.284+0000 7f19ee411740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c4a50fefa0 con 0x55c4a51de3e0 2024-09-16T02:07:11.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.284+0000 7f19ee411740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c4a51eba90 con 0x55c4a51ddac0 2024-09-16T02:07:11.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.284+0000 7f19ee411740 10 client.0 _ll_register_callbacks cb 0x55c4a4e52ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:11.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.285+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.285+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51e58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49094/0 (socket says 172.21.15.103:49094) 2024-09-16T02:07:11.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.285+0000 7f19e3e00640 1 -- 172.21.15.103:0/750031120 learned_addr learned my addr 172.21.15.103:0/750031120 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:11.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.285+0000 7f19e8e00640 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51ddac0 0x55c4a51e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.285+0000 7f19e9800640 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a51e5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.285+0000 7f19e3e00640 1 -- 172.21.15.103:0/750031120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51ddac0 msgr2=0x55c4a51e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:11.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.285+0000 7f19e3e00640 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51ddac0 0x55c4a51e85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.285+0000 7f19e3e00640 1 -- 172.21.15.103:0/750031120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 msgr2=0x55c4a51e5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:11.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.285+0000 7f19e3e00640 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a51e5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.287+0000 7f19e3e00640 1 -- 172.21.15.103:0/750031120 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a51e6650 con 0x55c4a51de3e0 2024-09-16T02:07:11.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.287+0000 7f19e3e00640 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51e58c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f19d000bd10 tx=0x7f19d0010a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3488a1731a1d3d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:11.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.287+0000 7f19caa00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:07:11.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.288+0000 7f19caa00640 1 -- 172.21.15.103:0/750031120 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0004430 con 0x55c4a51de3e0 2024-09-16T02:07:11.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.288+0000 7f19caa00640 1 -- 172.21.15.103:0/750031120 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f19d0005040 con 0x55c4a51de3e0 2024-09-16T02:07:11.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.288+0000 7f19ee411740 1 -- 172.21.15.103:0/750031120 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d0010d10 con 0x55c4a51de3e0 2024-09-16T02:07:11.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.289+0000 7f19caa00640 1 -- 172.21.15.103:0/750031120 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f19d0005240 con 0x55c4a51de3e0 2024-09-16T02:07:11.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.289+0000 7f19ee411740 10 client.25397 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:11.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.289+0000 7f19ee411740 1 -- 172.21.15.103:0/750031120 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c4a51f7850 con 0x55c4a51de3e0 2024-09-16T02:07:11.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.289+0000 7f19caa00640 1 -- 172.21.15.103:0/750031120 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19d0010d10 con 0x55c4a51de3e0 2024-09-16T02:07:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.289+0000 7f19caa00640 1 -- 172.21.15.103:0/750031120 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f19d00046c0 con 0x55c4a51de3e0 2024-09-16T02:07:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.290+0000 7f19ee411740 10 client.25397 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.290+0000 7f19ee411740 10 client.25397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.290+0000 7f19ee411740 20 client.25397 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.290+0000 7f19ee411740 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c4a51f8c50 0x55c4a51fb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.290+0000 7f19ee411740 4 client.25397 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.290+0000 7f19ee411740 1 -- 172.21.15.103:0/750031120 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4a5151800 con 0x55c4a51f8c50 2024-09-16T02:07:11.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.290+0000 7f19e8e00640 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c4a51f8c50 0x55c4a51fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.291+0000 7f19e8e00640 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c4a51f8c50 0x55c4a51fb040 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84dd6763b41be0f server_cookie=18f76e93551bcc66 in_seq=0 out_seq=0 2024-09-16T02:07:11.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.291+0000 7f19caa00640 10 client.25397 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.292+0000 7f19caa00640 1 -- 172.21.15.103:0/750031120 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4a5151800 con 0x55c4a51f8c50 2024-09-16T02:07:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.292+0000 7f19caa00640 10 client.25397 handle_command_reply: tid=0 2024-09-16T02:07:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.292+0000 7f19ee411740 1 client.25397 shutdown 2024-09-16T02:07:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 1 -- 172.21.15.103:0/750031120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 msgr2=0x55c4a51e58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51e58c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f19d000bd10 tx=0x7f19d0010a30 comp rx=0 tx=0).stop 2024-09-16T02:07:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 1 -- 172.21.15.103:0/750031120 shutdown_connections 2024-09-16T02:07:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c4a51f8c50 0x55c4a51fb040 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a51e5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51e58c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 1 --2- 172.21.15.103:0/750031120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51ddac0 0x55c4a51e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 1 -- 172.21.15.103:0/750031120 >> 172.21.15.103:0/750031120 conn(0x55c4a51da050 msgr2=0x55c4a51d91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19caa00640 0 client.25397 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 1 -- 172.21.15.103:0/750031120 shutdown_connections 2024-09-16T02:07:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 1 -- 172.21.15.103:0/750031120 wait complete. 2024-09-16T02:07:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.293+0000 7f19ee411740 20 client.25397 trim_cache size 0 max 16384 2024-09-16T02:07:11.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.314+0000 7f19ee411740 1 Processor -- start 2024-09-16T02:07:11.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.314+0000 7f19ee411740 1 -- start start 2024-09-16T02:07:11.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19ee411740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 0x55c4a51e0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19ee411740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 0x55c4a51e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19ee411740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a5200760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19ee411740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19e41ae030 con 0x55c4a51e8ca0 2024-09-16T02:07:11.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19ee411740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19e41ab900 con 0x55c4a51ddac0 2024-09-16T02:07:11.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19ee411740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19e41a9750 con 0x55c4a51de3e0 2024-09-16T02:07:11.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 0x55c4a51e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 0x55c4a51e7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:37898/0 (socket says 172.21.15.103:37898) 2024-09-16T02:07:11.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19e3e00640 1 -- 172.21.15.103:0/1610230053 learned_addr learned my addr 172.21.15.103:0/1610230053 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:11.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19e8e00640 1 --2- 172.21.15.103:0/1610230053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 0x55c4a51e0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19e9800640 1 --2- 172.21.15.103:0/1610230053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a5200760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19e3e00640 1 -- 172.21.15.103:0/1610230053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 msgr2=0x55c4a51e0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19e3e00640 1 --2- 172.21.15.103:0/1610230053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 0x55c4a51e0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19e3e00640 1 -- 172.21.15.103:0/1610230053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 msgr2=0x55c4a5200760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.315+0000 7f19e3e00640 1 --2- 172.21.15.103:0/1610230053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a5200760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.316+0000 7f19e3e00640 1 -- 172.21.15.103:0/1610230053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a51eb4f0 con 0x55c4a51de3e0 2024-09-16T02:07:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.316+0000 7f19e9800640 1 --2- 172.21.15.103:0/1610230053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a5200760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.316+0000 7f19e3e00640 1 --2- 172.21.15.103:0/1610230053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 0x55c4a51e7830 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19d0015040 tx=0x7f19d0012470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8739e1cdb580a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.316+0000 7f19d5600640 1 -- 172.21.15.103:0/1610230053 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d00053e0 con 0x55c4a51de3e0 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.316+0000 7f19d5600640 1 -- 172.21.15.103:0/1610230053 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19d0023070 con 0x55c4a51de3e0 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.316+0000 7f19d5600640 1 -- 172.21.15.103:0/1610230053 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0010aa0 con 0x55c4a51de3e0 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.317+0000 7f19ee411740 1 -- 172.21.15.103:0/1610230053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 msgr2=0x55c4a51e7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.317+0000 7f19ee411740 1 --2- 172.21.15.103:0/1610230053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 0x55c4a51e7830 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19d0015040 tx=0x7f19d0012470 comp rx=0 tx=0).stop 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.317+0000 7f19ee411740 1 -- 172.21.15.103:0/1610230053 shutdown_connections 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.317+0000 7f19ee411740 1 --2- 172.21.15.103:0/1610230053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51e8ca0 0x55c4a5200760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.317+0000 7f19ee411740 1 --2- 172.21.15.103:0/1610230053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51de3e0 0x55c4a51e7830 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.317+0000 7f19ee411740 1 --2- 172.21.15.103:0/1610230053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51ddac0 0x55c4a51e0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.317+0000 7f19ee411740 1 -- 172.21.15.103:0/1610230053 >> 172.21.15.103:0/1610230053 conn(0x55c4a51da050 msgr2=0x55c4a51e7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.317+0000 7f19ee411740 1 -- 172.21.15.103:0/1610230053 shutdown_connections 2024-09-16T02:07:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.317+0000 7f19ee411740 1 -- 172.21.15.103:0/1610230053 wait complete. 2024-09-16T02:07:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19ee411740 1 -- start start 2024-09-16T02:07:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19ee411740 1 Processor -- start 2024-09-16T02:07:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19ee411740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51ddac0 0x55c4a51f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19ee411740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51f1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19ee411740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51e8ca0 0x55c4a51edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19ee411740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19e41a9dc0 con 0x55c4a51ddac0 2024-09-16T02:07:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19ee411740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19e41ab230 con 0x55c4a51de3e0 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19ee411740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19e41aa490 con 0x55c4a51e8ca0 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19ee411740 10 client.0 _ll_register_callbacks cb 0x55c4a51e6820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51f1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49114/0 (socket says 172.21.15.103:49114) 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19e3e00640 1 -- 172.21.15.103:0/1012664170 learned_addr learned my addr 172.21.15.103:0/1012664170 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.318+0000 7f19e9800640 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51e8ca0 0x55c4a51edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19e8e00640 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51ddac0 0x55c4a51f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19e3e00640 1 -- 172.21.15.103:0/1012664170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51e8ca0 msgr2=0x55c4a51edcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19e3e00640 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51e8ca0 0x55c4a51edcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19e3e00640 1 -- 172.21.15.103:0/1012664170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51ddac0 msgr2=0x55c4a51f0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19e3e00640 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51ddac0 0x55c4a51f0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19e3e00640 1 -- 172.21.15.103:0/1012664170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a51ee1f0 con 0x55c4a51de3e0 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19e3e00640 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51f1220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f19d000bbd0 tx=0x7f19d0012870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca8b4611d1474ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19caa00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:07:11.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19caa00640 1 -- 172.21.15.103:0/1012664170 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d00056f0 con 0x55c4a51de3e0 2024-09-16T02:07:11.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19caa00640 1 -- 172.21.15.103:0/1012664170 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f19d0004cb0 con 0x55c4a51de3e0 2024-09-16T02:07:11.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.319+0000 7f19ee411740 1 -- 172.21.15.103:0/1012664170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d0023070 con 0x55c4a51de3e0 2024-09-16T02:07:11.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19caa00640 1 -- 172.21.15.103:0/1012664170 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f19d0049050 con 0x55c4a51de3e0 2024-09-16T02:07:11.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19ee411740 10 client.25403 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:11.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19ee411740 1 -- 172.21.15.103:0/1012664170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c4a5274e80 con 0x55c4a51de3e0 2024-09-16T02:07:11.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19caa00640 1 -- 172.21.15.103:0/1012664170 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19d0023070 con 0x55c4a51de3e0 2024-09-16T02:07:11.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19caa00640 1 -- 172.21.15.103:0/1012664170 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f19d0003a30 con 0x55c4a51de3e0 2024-09-16T02:07:11.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19ee411740 10 client.25403 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19ee411740 10 client.25403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19ee411740 20 client.25403 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19ee411740 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c4a5275560 0x55c4a5277980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19ee411740 4 client.25403 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19ee411740 1 -- 172.21.15.103:0/1012664170 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4a514bd10 con 0x55c4a5275560 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.320+0000 7f19e8e00640 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c4a5275560 0x55c4a5277980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.321+0000 7f19e8e00640 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c4a5275560 0x55c4a5277980 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c320f056428d8b9 server_cookie=6b191d9ccf6bde3 in_seq=0 out_seq=0 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.321+0000 7f19caa00640 10 client.25403 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.322+0000 7f19caa00640 1 -- 172.21.15.103:0/1012664170 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4a514bd10 con 0x55c4a5275560 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.322+0000 7f19caa00640 10 client.25403 handle_command_reply: tid=0 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.322+0000 7f19ee411740 1 client.25403 shutdown 2024-09-16T02:07:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.322+0000 7f19ee411740 1 -- 172.21.15.103:0/1012664170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 msgr2=0x55c4a51f1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.322+0000 7f19ee411740 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51f1220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f19d000bbd0 tx=0x7f19d0012870 comp rx=0 tx=0).stop 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.323+0000 7f19ee411740 1 -- 172.21.15.103:0/1012664170 shutdown_connections 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.323+0000 7f19ee411740 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55c4a5275560 0x55c4a5277980 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.323+0000 7f19ee411740 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c4a51e8ca0 0x55c4a51edcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.323+0000 7f19ee411740 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4a51de3e0 0x55c4a51f1220 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.323+0000 7f19ee411740 1 --2- 172.21.15.103:0/1012664170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a51ddac0 0x55c4a51f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.323+0000 7f19ee411740 1 -- 172.21.15.103:0/1012664170 >> 172.21.15.103:0/1012664170 conn(0x55c4a51da050 msgr2=0x55c4a51d91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.323+0000 7f19caa00640 0 client.25403 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.323+0000 7f19ee411740 1 -- 172.21.15.103:0/1012664170 shutdown_connections 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.323+0000 7f19ee411740 1 -- 172.21.15.103:0/1012664170 wait complete. 2024-09-16T02:07:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.323+0000 7f19ee411740 20 client.25403 trim_cache size 0 max 16384 2024-09-16T02:07:11.328 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:07:11.328 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-16T02:07:11.328 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "e01da340-43e7-4c63-9fd3-791d885d358a", 2024-09-16T02:07:11.328 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-16T02:07:11.328 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:07:11.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 -- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f19cc0a3300 msgr2=0x7f19cc0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:11.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f19cc0a3300 0x7f19cc0a57c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f19d8005c40 tx=0x7f19d802a9a0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 -- 172.21.15.103:0/2043006428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e410a2f0 msgr2=0x7f19e419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:11.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e410a2f0 0x7f19e419eff0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19dc013380 tx=0x7f19dc013840 comp rx=0 tx=0).stop 2024-09-16T02:07:11.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 -- 172.21.15.103:0/2043006428 shutdown_connections 2024-09-16T02:07:11.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f19cc0a3300 0x7f19cc0a57c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e410a2f0 0x7f19e419eff0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e4100780 0x7f19e419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 --2- 172.21.15.103:0/2043006428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19e40ffe60 0x7f19e419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:11.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 -- 172.21.15.103:0/2043006428 >> 172.21.15.103:0/2043006428 conn(0x7f19e40fba90 msgr2=0x7f19e4108df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:11.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 -- 172.21.15.103:0/2043006428 shutdown_connections 2024-09-16T02:07:11.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:11.324+0000 7f19eb800640 1 -- 172.21.15.103:0/2043006428 wait complete. 2024-09-16T02:07:11.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-16T02:07:11.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-09-16T02:07:11.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-16T02:07:11.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-16T02:07:11.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-09-16T02:07:11.564 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-16T02:07:11.565 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-09-16T02:07:11.566 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-16T02:07:11.567 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-09-16T02:07:12.195 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:07:12.297 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-16T02:07:12.300 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-09-16T02:07:12.300 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-16T02:07:12.300 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-09-16T02:07:12.300 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-09-16T02:07:12.301 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-16T02:07:12.302 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-09-16T02:07:12.303 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-09-16T02:07:12.304 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-16T02:07:12.305 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-09-16T02:07:12.306 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-16T02:07:12.306 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-09-16T02:07:12.306 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-16T02:07:12.306 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-09-16T02:07:12.306 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-16T02:07:12.306 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-09-16T02:07:12.306 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-16T02:07:12.306 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-09-16T02:07:12.306 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-16T02:07:12.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:12 smithi103 ceph-mon[30398]: scrub queued for path: / 2024-09-16T02:07:12.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:12 smithi103 ceph-mon[30398]: scrub summary: idle+waiting paths [/] 2024-09-16T02:07:12.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:12 smithi103 ceph-mon[30398]: scrub summary: active paths [/] 2024-09-16T02:07:12.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:12 smithi103 ceph-mon[30398]: scrub summary: idle+waiting paths [/] 2024-09-16T02:07:12.599 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:12.673 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-09-16T02:07:12.673 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-16T02:07:12.673 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-09-16T02:07:12.674 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-16T02:07:12.675 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-16T02:07:12.676 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-09-16T02:07:12.677 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-16T02:07:12.677 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-09-16T02:07:12.677 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-16T02:07:12.677 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-09-16T02:07:12.677 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-16T02:07:12.677 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-09-16T02:07:12.677 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-16T02:07:12.677 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-09-16T02:07:12.677 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-16T02:07:12.678 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-09-16T02:07:12.678 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-16T02:07:12.678 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-09-16T02:07:12.678 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-16T02:07:12.678 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-09-16T02:07:12.678 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-16T02:07:12.678 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-09-16T02:07:12.678 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-16T02:07:12.678 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-09-16T02:07:12.678 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-09-16T02:07:12.679 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-16T02:07:12.680 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-09-16T02:07:12.680 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-16T02:07:12.680 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-09-16T02:07:12.680 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-16T02:07:12.680 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-09-16T02:07:12.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:12 smithi136 ceph-mon[35323]: scrub queued for path: / 2024-09-16T02:07:12.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:12 smithi136 ceph-mon[35323]: scrub summary: idle+waiting paths [/] 2024-09-16T02:07:12.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:12 smithi136 ceph-mon[35323]: scrub summary: active paths [/] 2024-09-16T02:07:12.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:12 smithi136 ceph-mon[35323]: scrub summary: idle+waiting paths [/] 2024-09-16T02:07:12.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:12 smithi172 ceph-mon[35340]: scrub queued for path: / 2024-09-16T02:07:12.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:12 smithi172 ceph-mon[35340]: scrub summary: idle+waiting paths [/] 2024-09-16T02:07:12.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:12 smithi172 ceph-mon[35340]: scrub summary: active paths [/] 2024-09-16T02:07:12.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:12 smithi172 ceph-mon[35340]: scrub summary: idle+waiting paths [/] 2024-09-16T02:07:13.220 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-16T02:07:13.220 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-09-16T02:07:13.220 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-16T02:07:13.220 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-09-16T02:07:13.220 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-16T02:07:13.220 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-16T02:07:13.221 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-09-16T02:07:13.222 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-09-16T02:07:13.223 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-16T02:07:13.224 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-09-16T02:07:13.225 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-16T02:07:13.225 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-09-16T02:07:13.225 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-16T02:07:13.225 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-09-16T02:07:13.225 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-16T02:07:13.225 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-09-16T02:07:13.225 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-16T02:07:13.225 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-09-16T02:07:13.225 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-16T02:07:13.225 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-09-16T02:07:13.226 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-16T02:07:13.226 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-09-16T02:07:13.226 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-16T02:07:13.226 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-09-16T02:07:13.226 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-16T02:07:13.228 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-09-16T02:07:13.228 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-16T02:07:13.228 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-09-16T02:07:13.228 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-16T02:07:13.228 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-09-16T02:07:13.228 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.221+0000 7fc42aa00640 1 -- 172.21.15.103:0/3309811748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4241019d0 msgr2=0x7fc424101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.221+0000 7fc42aa00640 1 --2- 172.21.15.103:0/3309811748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4241019d0 0x7fc424101e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc410009930 tx=0x7fc41002a420 comp rx=0 tx=0).stop 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.221+0000 7fc42aa00640 1 -- 172.21.15.103:0/3309811748 shutdown_connections 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.221+0000 7fc42aa00640 1 --2- 172.21.15.103:0/3309811748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc424102370 0x7fc42410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.221+0000 7fc42aa00640 1 --2- 172.21.15.103:0/3309811748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4241019d0 0x7fc424101e30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.221+0000 7fc42aa00640 1 --2- 172.21.15.103:0/3309811748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4241079d0 0x7fc424107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.221+0000 7fc42aa00640 1 -- 172.21.15.103:0/3309811748 >> 172.21.15.103:0/3309811748 conn(0x7fc4240fd710 msgr2=0x7fc4240ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.221+0000 7fc42aa00640 1 -- 172.21.15.103:0/3309811748 shutdown_connections 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.221+0000 7fc42aa00640 1 -- 172.21.15.103:0/3309811748 wait complete. 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.222+0000 7fc42aa00640 1 Processor -- start 2024-09-16T02:07:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.222+0000 7fc42aa00640 1 -- start start 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.222+0000 7fc42aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4241019d0 0x7fc4241960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.223+0000 7fc42aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc424102370 0x7fc4241965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.223+0000 7fc42aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4241079d0 0x7fc42419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.223+0000 7fc42aa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc42410fc10 con 0x7fc424102370 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.223+0000 7fc42aa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc42410fa90 con 0x7fc4241019d0 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.223+0000 7fc42aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc42410fd90 con 0x7fc4241079d0 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.223+0000 7fc428c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc424102370 0x7fc4241965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.223+0000 7fc429600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4241019d0 0x7fc4241960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.223+0000 7fc429600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4241019d0 0x7fc4241960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49138/0 (socket says 172.21.15.103:49138) 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.223+0000 7fc429600640 1 -- 172.21.15.103:0/2025283148 learned_addr learned my addr 172.21.15.103:0/2025283148 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.224+0000 7fc42a000640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4241079d0 0x7fc42419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.224+0000 7fc429600640 1 -- 172.21.15.103:0/2025283148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4241079d0 msgr2=0x7fc42419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.224+0000 7fc429600640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4241079d0 0x7fc42419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.224+0000 7fc429600640 1 -- 172.21.15.103:0/2025283148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc424102370 msgr2=0x7fc4241965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.224+0000 7fc429600640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc424102370 0x7fc4241965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.224+0000 7fc429600640 1 -- 172.21.15.103:0/2025283148 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42419b060 con 0x7fc4241019d0 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.224+0000 7fc428c00640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc424102370 0x7fc4241965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.224+0000 7fc429600640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4241019d0 0x7fc4241960b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d770 tx=0x7fc41800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.226+0000 7fc422000640 1 -- 172.21.15.103:0/2025283148 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418017070 con 0x7fc4241019d0 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.226+0000 7fc422000640 1 -- 172.21.15.103:0/2025283148 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc418005500 con 0x7fc4241019d0 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.226+0000 7fc42aa00640 1 -- 172.21.15.103:0/2025283148 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42419b350 con 0x7fc4241019d0 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.226+0000 7fc42aa00640 1 -- 172.21.15.103:0/2025283148 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4241a2c30 con 0x7fc4241019d0 2024-09-16T02:07:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.226+0000 7fc422000640 1 -- 172.21.15.103:0/2025283148 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418004e00 con 0x7fc4241019d0 2024-09-16T02:07:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.232+0000 7fc422000640 1 -- 172.21.15.103:0/2025283148 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc418010460 con 0x7fc4241019d0 2024-09-16T02:07:13.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.233+0000 7fc42e9f1740 1 Processor -- start 2024-09-16T02:07:13.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.233+0000 7fc42e9f1740 1 -- start start 2024-09-16T02:07:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.234+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 0x55783b5ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.234+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.234+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.234+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55783b5203e0 con 0x55783b5b9d50 2024-09-16T02:07:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.234+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55783b4c7d80 con 0x55783b5c4f30 2024-09-16T02:07:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.234+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55783b51c3a0 con 0x55783b5ba670 2024-09-16T02:07:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.234+0000 7fc422000640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc40c0a77e0 0x7fc40c0a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.234+0000 7fc428c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.234+0000 7fc422000640 1 -- 172.21.15.103:0/2025283148 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fc4180d02b0 con 0x7fc4241019d0 2024-09-16T02:07:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.234+0000 7fc428c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49142/0 (socket says 172.21.15.103:49142) 2024-09-16T02:07:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.235+0000 7fc428c00640 1 -- 172.21.15.103:0/37029011 learned_addr learned my addr 172.21.15.103:0/37029011 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.235+0000 7fc429600640 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.235+0000 7fc428c00640 1 -- 172.21.15.103:0/37029011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 msgr2=0x55783b5c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.235+0000 7fc428c00640 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.235+0000 7fc428c00640 1 -- 172.21.15.103:0/37029011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 msgr2=0x55783b5ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.236+0000 7fc42a000640 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 0x55783b5ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.236+0000 7fc428c00640 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 0x55783b5ba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.236+0000 7fc428c00640 1 -- 172.21.15.103:0/37029011 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55783b4e6ae0 con 0x55783b5c4f30 2024-09-16T02:07:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.236+0000 7fc429600640 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.236+0000 7fc42a000640 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 0x55783b5ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:07:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.236+0000 7fc428c00640 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5c7320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc410000c00 tx=0x7fc410004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73848bde37e89fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.237+0000 7fc416600640 1 -- 172.21.15.103:0/37029011 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410035040 con 0x55783b5c4f30 2024-09-16T02:07:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.237+0000 7fc416600640 1 -- 172.21.15.103:0/37029011 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc410005400 con 0x55783b5c4f30 2024-09-16T02:07:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.237+0000 7fc42a000640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc40c0a77e0 0x7fc40c0a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.238+0000 7fc416600640 1 -- 172.21.15.103:0/37029011 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410003b90 con 0x55783b5c4f30 2024-09-16T02:07:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.239+0000 7fc42e9f1740 1 -- 172.21.15.103:0/37029011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 msgr2=0x55783b5c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.239+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5c7320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc410000c00 tx=0x7fc410004610 comp rx=0 tx=0).stop 2024-09-16T02:07:13.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.239+0000 7fc42e9f1740 1 -- 172.21.15.103:0/37029011 shutdown_connections 2024-09-16T02:07:13.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.239+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5c7320 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.239+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.239+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/37029011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 0x55783b5ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.239+0000 7fc42e9f1740 1 -- 172.21.15.103:0/37029011 >> 172.21.15.103:0/37029011 conn(0x55783b5b62e0 msgr2=0x55783b5b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.240+0000 7fc42a000640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc40c0a77e0 0x7fc40c0a9ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc41c005c40 tx=0x7fc41c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.240+0000 7fc42e9f1740 1 -- 172.21.15.103:0/37029011 shutdown_connections 2024-09-16T02:07:13.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.241+0000 7fc42e9f1740 1 -- 172.21.15.103:0/37029011 wait complete. 2024-09-16T02:07:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.243+0000 7fc42e9f1740 1 -- start start 2024-09-16T02:07:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.244+0000 7fc42e9f1740 1 Processor -- start 2024-09-16T02:07:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5b9d50 0x55783b5c1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5c4f30 0x55783b5c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55783b5c7c40 con 0x55783b5c4f30 2024-09-16T02:07:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55783b4db0f0 con 0x55783b5b9d50 2024-09-16T02:07:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55783b5c7dc0 con 0x55783b5ba670 2024-09-16T02:07:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc42e9f1740 10 client.0 _ll_register_callbacks cb 0x55783b4cc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc42a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5c4f30 0x55783b5c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc42a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5c4f30 0x55783b5c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35998/0 (socket says 172.21.15.103:35998) 2024-09-16T02:07:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc42a000640 1 -- 172.21.15.103:0/2550986032 learned_addr learned my addr 172.21.15.103:0/2550986032 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.247+0000 7fc429600640 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5b9d50 0x55783b5c1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.247+0000 7fc42a000640 1 -- 172.21.15.103:0/2550986032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 msgr2=0x55783b5c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.246+0000 7fc428c00640 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.247+0000 7fc42a000640 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.247+0000 7fc42a000640 1 -- 172.21.15.103:0/2550986032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5b9d50 msgr2=0x55783b5c1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.247+0000 7fc42a000640 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5b9d50 0x55783b5c1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.247+0000 7fc42a000640 1 -- 172.21.15.103:0/2550986032 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55783b2913c0 con 0x55783b5c4f30 2024-09-16T02:07:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.247+0000 7fc429600640 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5b9d50 0x55783b5c1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:07:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.247+0000 7fc428c00640 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:07:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.247+0000 7fc42a000640 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5c4f30 0x55783b5c2760 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc41c010e30 tx=0x7fc41c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ceb6ae333dde1b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.248+0000 7fc40aa00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:07:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.248+0000 7fc40aa00640 1 -- 172.21.15.103:0/2550986032 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c026680 con 0x55783b5c4f30 2024-09-16T02:07:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.248+0000 7fc40aa00640 1 -- 172.21.15.103:0/2550986032 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fc41c026480 con 0x55783b5c4f30 2024-09-16T02:07:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.248+0000 7fc42e9f1740 1 -- 172.21.15.103:0/2550986032 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc410005400 con 0x55783b5c4f30 2024-09-16T02:07:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.248+0000 7fc40aa00640 1 -- 172.21.15.103:0/2550986032 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc41c026680 con 0x55783b5c4f30 2024-09-16T02:07:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.248+0000 7fc42e9f1740 10 client.15435 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.248+0000 7fc42e9f1740 1 -- 172.21.15.103:0/2550986032 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55783b5dc090 con 0x55783b5c4f30 2024-09-16T02:07:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.249+0000 7fc40aa00640 1 -- 172.21.15.103:0/2550986032 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc410005400 con 0x55783b5c4f30 2024-09-16T02:07:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.249+0000 7fc40aa00640 1 -- 172.21.15.103:0/2550986032 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc41c030910 con 0x55783b5c4f30 2024-09-16T02:07:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.249+0000 7fc42e9f1740 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.249+0000 7fc42e9f1740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.249+0000 7fc42e9f1740 20 client.15435 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:13.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.249+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55783b5dd540 0x55783b5df990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.249+0000 7fc42e9f1740 4 client.15435 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:13.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.249+0000 7fc42e9f1740 1 -- 172.21.15.103:0/2550986032 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55783b52d950 con 0x55783b5dd540 2024-09-16T02:07:13.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.250+0000 7fc429600640 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55783b5dd540 0x55783b5df990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.250+0000 7fc40aa00640 10 client.15435 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:13.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.252+0000 7fc429600640 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55783b5dd540 0x55783b5df990 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e60da4637d108 server_cookie=4267e213dab89ed9 in_seq=0 out_seq=0 2024-09-16T02:07:13.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.253+0000 7fc40aa00640 1 -- 172.21.15.103:0/2550986032 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55783b52d950 con 0x55783b5dd540 2024-09-16T02:07:13.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.253+0000 7fc40aa00640 10 client.15435 handle_command_reply: tid=0 2024-09-16T02:07:13.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.254+0000 7fc42e9f1740 1 client.15435 shutdown 2024-09-16T02:07:13.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.254+0000 7fc42e9f1740 1 -- 172.21.15.103:0/2550986032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5c4f30 msgr2=0x55783b5c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.254+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5c4f30 0x55783b5c2760 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc41c010e30 tx=0x7fc41c004ab0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.255+0000 7fc42e9f1740 1 -- 172.21.15.103:0/2550986032 shutdown_connections 2024-09-16T02:07:13.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.255+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55783b5dd540 0x55783b5df990 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.255+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5c4f30 0x55783b5c2760 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.255+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.255+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/2550986032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5b9d50 0x55783b5c1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.255+0000 7fc42e9f1740 1 -- 172.21.15.103:0/2550986032 >> 172.21.15.103:0/2550986032 conn(0x55783b5b62e0 msgr2=0x55783b5bc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.255+0000 7fc40aa00640 0 client.15435 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:13.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.255+0000 7fc42e9f1740 1 -- 172.21.15.103:0/2550986032 shutdown_connections 2024-09-16T02:07:13.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.256+0000 7fc42e9f1740 1 -- 172.21.15.103:0/2550986032 wait complete. 2024-09-16T02:07:13.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.256+0000 7fc42e9f1740 20 client.15435 trim_cache size 0 max 16384 2024-09-16T02:07:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.286+0000 7fc42e9f1740 1 Processor -- start 2024-09-16T02:07:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.287+0000 7fc42e9f1740 1 -- start start 2024-09-16T02:07:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.287+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5b9d50 0x55783b5e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.287+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5ba670 0x55783b5e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.287+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.287+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc404011000 con 0x55783b5ba670 2024-09-16T02:07:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.287+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc40400e8d0 con 0x55783b5c4f30 2024-09-16T02:07:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.287+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc40400c720 con 0x55783b5b9d50 2024-09-16T02:07:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.287+0000 7fc42a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.287+0000 7fc42a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49148/0 (socket says 172.21.15.103:49148) 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.287+0000 7fc42a000640 1 -- 172.21.15.103:0/243049412 learned_addr learned my addr 172.21.15.103:0/243049412 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.288+0000 7fc42a000640 1 -- 172.21.15.103:0/243049412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5b9d50 msgr2=0x55783b5e52a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.288+0000 7fc42a000640 1 --2- 172.21.15.103:0/243049412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5b9d50 0x55783b5e52a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.288+0000 7fc42a000640 1 -- 172.21.15.103:0/243049412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5ba670 msgr2=0x55783b5e57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.288+0000 7fc42a000640 1 --2- 172.21.15.103:0/243049412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5ba670 0x55783b5e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.288+0000 7fc42a000640 1 -- 172.21.15.103:0/243049412 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55783b5b6110 con 0x55783b5c4f30 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.288+0000 7fc42a000640 1 --2- 172.21.15.103:0/243049412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bb7f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc41c010e00 tx=0x7fc41c005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ab24670b1bf3f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.289+0000 7fc415c00640 1 -- 172.21.15.103:0/243049412 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00a2d0 con 0x55783b5c4f30 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.289+0000 7fc415c00640 1 -- 172.21.15.103:0/243049412 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41c026d00 con 0x55783b5c4f30 2024-09-16T02:07:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.289+0000 7fc415c00640 1 -- 172.21.15.103:0/243049412 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c01d680 con 0x55783b5c4f30 2024-09-16T02:07:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.290+0000 7fc42e9f1740 1 -- 172.21.15.103:0/243049412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 msgr2=0x55783b5bb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.290+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/243049412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bb7f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc41c010e00 tx=0x7fc41c005f60 comp rx=0 tx=0).stop 2024-09-16T02:07:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.290+0000 7fc42e9f1740 1 -- 172.21.15.103:0/243049412 shutdown_connections 2024-09-16T02:07:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.290+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/243049412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bb7f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.290+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/243049412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5ba670 0x55783b5e57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.290+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/243049412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5b9d50 0x55783b5e52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.290+0000 7fc42e9f1740 1 -- 172.21.15.103:0/243049412 >> 172.21.15.103:0/243049412 conn(0x55783b5b62e0 msgr2=0x55783b5c0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.290+0000 7fc42e9f1740 1 -- 172.21.15.103:0/243049412 shutdown_connections 2024-09-16T02:07:13.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.290+0000 7fc42e9f1740 1 -- 172.21.15.103:0/243049412 wait complete. 2024-09-16T02:07:13.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.291+0000 7fc42e9f1740 1 -- start start 2024-09-16T02:07:13.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.291+0000 7fc42e9f1740 1 Processor -- start 2024-09-16T02:07:13.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.292+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 0x55783b5cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.292+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5bea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.292+0000 7fc42e9f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.292+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc40400cd90 con 0x55783b5b9d50 2024-09-16T02:07:13.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.292+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc40400e200 con 0x55783b5c4f30 2024-09-16T02:07:13.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.292+0000 7fc42e9f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc40400d460 con 0x55783b5ba670 2024-09-16T02:07:13.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.292+0000 7fc42e9f1740 10 client.0 _ll_register_callbacks cb 0x7fc41c010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:13.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.293+0000 7fc428c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.293+0000 7fc42a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.293+0000 7fc42a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49162/0 (socket says 172.21.15.103:49162) 2024-09-16T02:07:13.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.293+0000 7fc42a000640 1 -- 172.21.15.103:0/3142038044 learned_addr learned my addr 172.21.15.103:0/3142038044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:13.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.294+0000 7fc429600640 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 0x55783b5cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.294+0000 7fc428c00640 1 -- 172.21.15.103:0/3142038044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 msgr2=0x55783b5bef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.294+0000 7fc428c00640 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.294+0000 7fc428c00640 1 -- 172.21.15.103:0/3142038044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 msgr2=0x55783b5cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.294+0000 7fc428c00640 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 0x55783b5cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.294+0000 7fc428c00640 1 -- 172.21.15.103:0/3142038044 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55783b5bf700 con 0x55783b5ba670 2024-09-16T02:07:13.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.294+0000 7fc42a000640 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:13.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.294+0000 7fc429600640 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 0x55783b5cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:13.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.295+0000 7fc428c00640 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5bea50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55783b5c3320 tx=0x7fc410005430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf6c0b16df1fbfa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.295+0000 7fc40aa00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:07:13.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.295+0000 7fc40aa00640 1 -- 172.21.15.103:0/3142038044 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410035040 con 0x55783b5ba670 2024-09-16T02:07:13.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.296+0000 7fc42e9f1740 1 -- 172.21.15.103:0/3142038044 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc41c026d00 con 0x55783b5ba670 2024-09-16T02:07:13.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.296+0000 7fc40aa00640 1 -- 172.21.15.103:0/3142038044 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fc410004700 con 0x55783b5ba670 2024-09-16T02:07:13.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.296+0000 7fc40aa00640 1 -- 172.21.15.103:0/3142038044 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc410004900 con 0x55783b5ba670 2024-09-16T02:07:13.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.296+0000 7fc42e9f1740 10 client.25390 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:13.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.296+0000 7fc42e9f1740 1 -- 172.21.15.103:0/3142038044 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55783b648270 con 0x55783b5ba670 2024-09-16T02:07:13.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.296+0000 7fc40aa00640 1 -- 172.21.15.103:0/3142038044 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41c026d00 con 0x55783b5ba670 2024-09-16T02:07:13.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.297+0000 7fc40aa00640 1 -- 172.21.15.103:0/3142038044 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc410003c00 con 0x55783b5ba670 2024-09-16T02:07:13.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.297+0000 7fc42e9f1740 10 client.25390 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:13.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.297+0000 7fc42e9f1740 10 client.25390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:13.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.297+0000 7fc42e9f1740 20 client.25390 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:13.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.297+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55783b648500 0x55783b64a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.297+0000 7fc42e9f1740 4 client.25390 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:07:13.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.297+0000 7fc42e9f1740 1 -- 172.21.15.103:0/3142038044 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55783b527e60 con 0x55783b648500 2024-09-16T02:07:13.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.299+0000 7fc429600640 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55783b648500 0x55783b64a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.300+0000 7fc429600640 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55783b648500 0x55783b64a8f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b727f056618d57c server_cookie=f580c9f3de960a7e in_seq=0 out_seq=0 2024-09-16T02:07:13.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.300+0000 7fc40aa00640 10 client.25390 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:13.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.300+0000 7fc40aa00640 1 -- 172.21.15.103:0/3142038044 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55783b527e60 con 0x55783b648500 2024-09-16T02:07:13.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.301+0000 7fc40aa00640 10 client.25390 handle_command_reply: tid=0 2024-09-16T02:07:13.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.301+0000 7fc42e9f1740 1 client.25390 shutdown 2024-09-16T02:07:13.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.301+0000 7fc42e9f1740 1 -- 172.21.15.103:0/3142038044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 msgr2=0x55783b5bea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.301+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5bea50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55783b5c3320 tx=0x7fc410005430 comp rx=0 tx=0).stop 2024-09-16T02:07:13.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.302+0000 7fc42e9f1740 1 -- 172.21.15.103:0/3142038044 shutdown_connections 2024-09-16T02:07:13.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.302+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55783b648500 0x55783b64a8f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.302+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55783b5c4f30 0x55783b5bef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.302+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55783b5ba670 0x55783b5bea50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.302+0000 7fc42e9f1740 1 --2- 172.21.15.103:0/3142038044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55783b5b9d50 0x55783b5cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.302+0000 7fc40aa00640 0 client.25390 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:13.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.302+0000 7fc42e9f1740 1 -- 172.21.15.103:0/3142038044 >> 172.21.15.103:0/3142038044 conn(0x55783b5b62e0 msgr2=0x55783b5bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.302+0000 7fc42e9f1740 1 -- 172.21.15.103:0/3142038044 shutdown_connections 2024-09-16T02:07:13.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.303+0000 7fc42e9f1740 1 -- 172.21.15.103:0/3142038044 wait complete. 2024-09-16T02:07:13.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.303+0000 7fc42e9f1740 20 client.25390 trim_cache size 0 max 16384 2024-09-16T02:07:13.307 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:07:13.308 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T02:07:13.308 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-16T02:07:13.308 INFO:teuthology.orchestra.run.smithi103.stdout: "e01da340-43e7-4c63-9fd3-791d885d358a": { 2024-09-16T02:07:13.308 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-16T02:07:13.308 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "e01da340-43e7-4c63-9fd3-791d885d358a", 2024-09-16T02:07:13.309 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-16T02:07:13.309 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:13.309 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:13.310 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:07:13.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 -- 172.21.15.103:0/2025283148 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc40c0a77e0 msgr2=0x7fc40c0a9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc40c0a77e0 0x7fc40c0a9ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc41c005c40 tx=0x7fc41c004a40 comp rx=0 tx=0).stop 2024-09-16T02:07:13.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 -- 172.21.15.103:0/2025283148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4241019d0 msgr2=0x7fc4241960b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4241019d0 0x7fc4241960b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d770 tx=0x7fc41800dc30 comp rx=0 tx=0).stop 2024-09-16T02:07:13.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 -- 172.21.15.103:0/2025283148 shutdown_connections 2024-09-16T02:07:13.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fc40c0a77e0 0x7fc40c0a9ca0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4241079d0 0x7fc42419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc424102370 0x7fc4241965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 --2- 172.21.15.103:0/2025283148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4241019d0 0x7fc4241960b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 -- 172.21.15.103:0/2025283148 >> 172.21.15.103:0/2025283148 conn(0x7fc4240fd710 msgr2=0x7fc4240fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.307+0000 7fc42aa00640 1 -- 172.21.15.103:0/2025283148 shutdown_connections 2024-09-16T02:07:13.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:13.308+0000 7fc42aa00640 1 -- 172.21.15.103:0/2025283148 wait complete. 2024-09-16T02:07:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.600+0000 7f0426600640 1 -- 172.21.15.103:0/3930379753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0420105910 msgr2=0x7f042010c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.600+0000 7f0426600640 1 --2- 172.21.15.103:0/3930379753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0420105910 0x7f042010c1a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0418009ca0 tx=0x7f041802a8c0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.600+0000 7f0426600640 1 -- 172.21.15.103:0/3930379753 shutdown_connections 2024-09-16T02:07:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.600+0000 7f0426600640 1 --2- 172.21.15.103:0/3930379753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0420105910 0x7f042010c1a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.600+0000 7f0426600640 1 --2- 172.21.15.103:0/3930379753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420104f50 0x7f04201053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.600+0000 7f0426600640 1 --2- 172.21.15.103:0/3930379753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0420103d60 0x7f0420104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.600+0000 7f0426600640 1 -- 172.21.15.103:0/3930379753 >> 172.21.15.103:0/3930379753 conn(0x7f04200ff4e0 msgr2=0x7f0420101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.600+0000 7f0426600640 1 -- 172.21.15.103:0/3930379753 shutdown_connections 2024-09-16T02:07:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.600+0000 7f0426600640 1 -- 172.21.15.103:0/3930379753 wait complete. 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f0426600640 1 Processor -- start 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f0426600640 1 -- start start 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f0426600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420103d60 0x7f0420199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f0426600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0420104f50 0x7f042019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f0426600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0420105910 0x7f04201a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f0426600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f042010fa30 con 0x7f0420105910 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f0426600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f042010f8b0 con 0x7f0420103d60 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f0426600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f042010fbb0 con 0x7f0420104f50 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f041ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0420104f50 0x7f042019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f041f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420103d60 0x7f0420199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.601+0000 7f041ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0420104f50 0x7f042019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:37972/0 (socket says 172.21.15.103:37972) 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f041ea00640 1 -- 172.21.15.103:0/3418748508 learned_addr learned my addr 172.21.15.103:0/3418748508 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f041fe00640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0420105910 0x7f04201a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f041ea00640 1 -- 172.21.15.103:0/3418748508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420103d60 msgr2=0x7f0420199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f041ea00640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420103d60 0x7f0420199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f041ea00640 1 -- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0420105910 msgr2=0x7f04201a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f041ea00640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0420105910 0x7f04201a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f041ea00640 1 -- 172.21.15.103:0/3418748508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04201a1b80 con 0x7f0420104f50 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f041fe00640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0420105910 0x7f04201a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f041ea00640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0420104f50 0x7f042019a4a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e9f0 tx=0x7f040c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f0417e00640 1 -- 172.21.15.103:0/3418748508 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c00cd80 con 0x7f0420104f50 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f0417e00640 1 -- 172.21.15.103:0/3418748508 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f040c005500 con 0x7f0420104f50 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f0426600640 1 -- 172.21.15.103:0/3418748508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04201a1e70 con 0x7f0420104f50 2024-09-16T02:07:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.602+0000 7f0417e00640 1 -- 172.21.15.103:0/3418748508 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c010690 con 0x7f0420104f50 2024-09-16T02:07:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.603+0000 7f0426600640 1 -- 172.21.15.103:0/3418748508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04201a2330 con 0x7f0420104f50 2024-09-16T02:07:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f0429122740 1 Processor -- start 2024-09-16T02:07:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f0429122740 1 -- start start 2024-09-16T02:07:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f0429122740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 0x56041a3f0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f0429122740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 0x56041a3f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f0429122740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a3f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f0429122740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56041a355100 con 0x56041a3f09d0 2024-09-16T02:07:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f0429122740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56041a2fd1b0 con 0x56041a3f5a60 2024-09-16T02:07:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f0429122740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56041a3510c0 con 0x56041a3f0090 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f041fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 0x56041a3f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f041fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 0x56041a3f0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:37978/0 (socket says 172.21.15.103:37978) 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.607+0000 7f041fe00640 1 -- 172.21.15.103:0/3797391441 learned_addr learned my addr 172.21.15.103:0/3797391441 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.608+0000 7f041f400640 1 --2- 172.21.15.103:0/3797391441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 0x56041a3f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.608+0000 7f0417e00640 1 -- 172.21.15.103:0/3418748508 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f040c0046c0 con 0x7f0420104f50 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.608+0000 7f041fe00640 1 -- 172.21.15.103:0/3797391441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 msgr2=0x56041a3f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.608+0000 7f041fe00640 1 --2- 172.21.15.103:0/3797391441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a3f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.608+0000 7f041fe00640 1 -- 172.21.15.103:0/3797391441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 msgr2=0x56041a3f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.608+0000 7f041fe00640 1 --2- 172.21.15.103:0/3797391441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 0x56041a3f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.608+0000 7f041fe00640 1 -- 172.21.15.103:0/3797391441 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56041a068210 con 0x56041a3f0090 2024-09-16T02:07:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.608+0000 7f041f400640 1 --2- 172.21.15.103:0/3797391441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 0x56041a3f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.608+0000 7f041fe00640 1 --2- 172.21.15.103:0/3797391441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 0x56041a3f0490 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0418005e30 tx=0x7f0418005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b09459e074e29eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.609+0000 7f0414c00640 1 -- 172.21.15.103:0/3797391441 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418031650 con 0x56041a3f0090 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.609+0000 7f0414c00640 1 -- 172.21.15.103:0/3797391441 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0418036040 con 0x56041a3f0090 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.609+0000 7f0414c00640 1 -- 172.21.15.103:0/3797391441 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418004760 con 0x56041a3f0090 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.609+0000 7f0417e00640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f04040a3370 0x7f04040a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.609+0000 7f0417e00640 1 -- 172.21.15.103:0/3418748508 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f040c0cfd90 con 0x7f0420104f50 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.609+0000 7f041f400640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f04040a3370 0x7f04040a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.609+0000 7f0429122740 1 -- 172.21.15.103:0/3797391441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 msgr2=0x56041a3f0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.610+0000 7f0429122740 1 --2- 172.21.15.103:0/3797391441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 0x56041a3f0490 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0418005e30 tx=0x7f0418005f70 comp rx=0 tx=0).stop 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.610+0000 7f0429122740 1 -- 172.21.15.103:0/3797391441 shutdown_connections 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.610+0000 7f0429122740 1 --2- 172.21.15.103:0/3797391441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a3f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.610+0000 7f0429122740 1 --2- 172.21.15.103:0/3797391441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 0x56041a3f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.610+0000 7f0429122740 1 --2- 172.21.15.103:0/3797391441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 0x56041a3f0490 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.610+0000 7f0429122740 1 -- 172.21.15.103:0/3797391441 >> 172.21.15.103:0/3797391441 conn(0x56041a3eb700 msgr2=0x56041a3ebb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.610+0000 7f0429122740 1 -- 172.21.15.103:0/3797391441 shutdown_connections 2024-09-16T02:07:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.610+0000 7f0429122740 1 -- 172.21.15.103:0/3797391441 wait complete. 2024-09-16T02:07:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.610+0000 7f041f400640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f04040a3370 0x7f04040a5830 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56041a3ec580 tx=0x7f0410005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.610+0000 7f0429122740 1 -- start start 2024-09-16T02:07:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f0429122740 1 Processor -- start 2024-09-16T02:07:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f0429122740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 0x56041a3fa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f0429122740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f09d0 0x56041a3fa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f0429122740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f5a60 0x56041a4017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f0429122740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56041a3f89d0 con 0x56041a3f0090 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f0429122740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56041a2de090 con 0x56041a3f09d0 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f0429122740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56041a3f8b50 con 0x56041a3f5a60 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f0429122740 10 client.0 _ll_register_callbacks cb 0x56041a332f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f041ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f09d0 0x56041a3fa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f041ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f09d0 0x56041a3fa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49214/0 (socket says 172.21.15.103:49214) 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f041f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 0x56041a3fa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.611+0000 7f041ea00640 1 -- 172.21.15.103:0/2550741047 learned_addr learned my addr 172.21.15.103:0/2550741047 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f041fe00640 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f5a60 0x56041a4017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f041ea00640 1 -- 172.21.15.103:0/2550741047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f5a60 msgr2=0x56041a4017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f041ea00640 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f5a60 0x56041a4017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f041ea00640 1 -- 172.21.15.103:0/2550741047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 msgr2=0x56041a3fa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f041ea00640 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 0x56041a3fa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f041ea00640 1 -- 172.21.15.103:0/2550741047 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56041a134620 con 0x56041a3f09d0 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f041fe00640 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f5a60 0x56041a4017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f041ea00640 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f09d0 0x56041a3fa780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f040c09c030 tx=0x7f040c09bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26f829ee416874e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f0409600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f0409600640 1 -- 172.21.15.103:0/2550741047 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0a4d10 con 0x56041a3f09d0 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f0409600640 1 -- 172.21.15.103:0/2550741047 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f040c0b5210 con 0x56041a3f09d0 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.612+0000 7f0429122740 1 -- 172.21.15.103:0/2550741047 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0418036040 con 0x56041a3f09d0 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0409600640 1 -- 172.21.15.103:0/2550741047 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f040c0a06e0 con 0x56041a3f09d0 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0429122740 10 client.25427 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0429122740 1 -- 172.21.15.103:0/2550741047 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56041a404680 con 0x56041a3f09d0 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0409600640 1 -- 172.21.15.103:0/2550741047 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0418036040 con 0x56041a3f09d0 2024-09-16T02:07:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0409600640 1 -- 172.21.15.103:0/2550741047 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f040c098110 con 0x56041a3f09d0 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0429122740 10 client.25427 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0429122740 10 client.25427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0429122740 20 client.25427 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0429122740 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x56041a405d50 0x56041a408140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0429122740 4 client.25427 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.613+0000 7f0429122740 1 -- 172.21.15.103:0/2550741047 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56041a362830 con 0x56041a405d50 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.614+0000 7f041f400640 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x56041a405d50 0x56041a408140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.614+0000 7f041f400640 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x56041a405d50 0x56041a408140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4078f11c89acb7 server_cookie=8cba379b784b0158 in_seq=0 out_seq=0 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.614+0000 7f0409600640 10 client.25427 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0409600640 1 -- 172.21.15.103:0/2550741047 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56041a362830 con 0x56041a405d50 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0409600640 10 client.25427 handle_command_reply: tid=0 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 client.25427 shutdown 2024-09-16T02:07:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 -- 172.21.15.103:0/2550741047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f09d0 msgr2=0x56041a3fa780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f09d0 0x56041a3fa780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f040c09c030 tx=0x7f040c09bd50 comp rx=0 tx=0).stop 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 -- 172.21.15.103:0/2550741047 shutdown_connections 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x56041a405d50 0x56041a408140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f5a60 0x56041a4017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0409600640 0 client.25427 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f09d0 0x56041a3fa780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 --2- 172.21.15.103:0/2550741047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 0x56041a3fa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 -- 172.21.15.103:0/2550741047 >> 172.21.15.103:0/2550741047 conn(0x56041a3eb700 msgr2=0x56041a3ecf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 -- 172.21.15.103:0/2550741047 shutdown_connections 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.615+0000 7f0429122740 1 -- 172.21.15.103:0/2550741047 wait complete. 2024-09-16T02:07:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.616+0000 7f0429122740 20 client.25427 trim_cache size 0 max 16384 2024-09-16T02:07:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.631+0000 7f0429122740 1 Processor -- start 2024-09-16T02:07:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.631+0000 7f0429122740 1 -- start start 2024-09-16T02:07:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.631+0000 7f0429122740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 0x56041a3ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.631+0000 7f0429122740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 0x56041a401780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.631+0000 7f0429122740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a3ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.631+0000 7f0429122740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04201adc00 con 0x56041a3f09d0 2024-09-16T02:07:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.631+0000 7f0429122740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f04201ab4d0 con 0x56041a3f5a60 2024-09-16T02:07:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.631+0000 7f0429122740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04201a9320 con 0x56041a3f0090 2024-09-16T02:07:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 0x56041a401780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a3ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a3ecaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:49224/0 (socket says 172.21.15.103:49224) 2024-09-16T02:07:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041fe00640 1 -- 172.21.15.103:0/1261147409 learned_addr learned my addr 172.21.15.103:0/1261147409 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041f400640 1 --2- 172.21.15.103:0/1261147409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 0x56041a3ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041ea00640 1 -- 172.21.15.103:0/1261147409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 msgr2=0x56041a3ec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041ea00640 1 --2- 172.21.15.103:0/1261147409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 0x56041a3ec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041ea00640 1 -- 172.21.15.103:0/1261147409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 msgr2=0x56041a3ecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041ea00640 1 --2- 172.21.15.103:0/1261147409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a3ecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041ea00640 1 -- 172.21.15.103:0/1261147409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56041a4087a0 con 0x56041a3f09d0 2024-09-16T02:07:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041fe00640 1 --2- 172.21.15.103:0/1261147409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a3ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.632+0000 7f041ea00640 1 --2- 172.21.15.103:0/1261147409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 0x56041a401780 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e9c0 tx=0x7f040c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a49619a862998f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.633+0000 7f040be00640 1 -- 172.21.15.103:0/1261147409 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0b5290 con 0x56041a3f09d0 2024-09-16T02:07:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.633+0000 7f040be00640 1 -- 172.21.15.103:0/1261147409 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f040c0a4fa0 con 0x56041a3f09d0 2024-09-16T02:07:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.633+0000 7f040be00640 1 -- 172.21.15.103:0/1261147409 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c09b780 con 0x56041a3f09d0 2024-09-16T02:07:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.633+0000 7f0429122740 1 -- 172.21.15.103:0/1261147409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 msgr2=0x56041a401780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.633+0000 7f0429122740 1 --2- 172.21.15.103:0/1261147409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 0x56041a401780 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e9c0 tx=0x7f040c004a40 comp rx=0 tx=0).stop 2024-09-16T02:07:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.634+0000 7f0429122740 1 -- 172.21.15.103:0/1261147409 shutdown_connections 2024-09-16T02:07:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.634+0000 7f0429122740 1 --2- 172.21.15.103:0/1261147409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a3ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.634+0000 7f0429122740 1 --2- 172.21.15.103:0/1261147409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f09d0 0x56041a401780 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.634+0000 7f0429122740 1 --2- 172.21.15.103:0/1261147409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f0090 0x56041a3ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.634+0000 7f0429122740 1 -- 172.21.15.103:0/1261147409 >> 172.21.15.103:0/1261147409 conn(0x56041a3eb700 msgr2=0x56041a3f1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.634+0000 7f0429122740 1 -- 172.21.15.103:0/1261147409 shutdown_connections 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.634+0000 7f0429122740 1 -- 172.21.15.103:0/1261147409 wait complete. 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.634+0000 7f0429122740 1 -- start start 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.634+0000 7f0429122740 1 Processor -- start 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f0429122740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 0x56041a3ff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f0429122740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f09d0 0x56041a3ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f0429122740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a400680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f0429122740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04201a9990 con 0x56041a3f0090 2024-09-16T02:07:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f0429122740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f04201aae00 con 0x56041a3f5a60 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f0429122740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04201aa060 con 0x56041a3f09d0 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f0429122740 10 client.0 _ll_register_callbacks cb 0x7f040c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 0x56041a3ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a400680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 0x56041a3ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36068/0 (socket says 172.21.15.103:36068) 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041f400640 1 -- 172.21.15.103:0/1196730428 learned_addr learned my addr 172.21.15.103:0/1196730428 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041ea00640 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f09d0 0x56041a3ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041f400640 1 -- 172.21.15.103:0/1196730428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f09d0 msgr2=0x56041a3ff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041f400640 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f09d0 0x56041a3ff6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041f400640 1 -- 172.21.15.103:0/1196730428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 msgr2=0x56041a400680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041f400640 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a400680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041f400640 1 -- 172.21.15.103:0/1196730428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56041a400c90 con 0x56041a3f0090 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.635+0000 7f041fe00640 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a400680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.636+0000 7f041f400640 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 0x56041a3ff180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x56041a3f1400 tx=0x7f04100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b1fe34fc88ef26 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.636+0000 7f0409600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.636+0000 7f0409600640 1 -- 172.21.15.103:0/1196730428 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041001dce0 con 0x56041a3f0090 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.636+0000 7f0429122740 1 -- 172.21.15.103:0/1196730428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040c0a4fa0 con 0x56041a3f0090 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.636+0000 7f0409600640 1 -- 172.21.15.103:0/1196730428 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f0410005020 con 0x56041a3f0090 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.636+0000 7f0409600640 1 -- 172.21.15.103:0/1196730428 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0410038030 con 0x56041a3f0090 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.636+0000 7f0429122740 10 client.15456 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.636+0000 7f0429122740 1 -- 172.21.15.103:0/1196730428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56041a470d40 con 0x56041a3f0090 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.636+0000 7f0409600640 1 -- 172.21.15.103:0/1196730428 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f040c0a4fa0 con 0x56041a3f0090 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.637+0000 7f0409600640 1 -- 172.21.15.103:0/1196730428 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0410033480 con 0x56041a3f0090 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.637+0000 7f0429122740 10 client.15456 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.637+0000 7f0429122740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.637+0000 7f0429122740 20 client.15456 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.637+0000 7f0429122740 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x56041a471640 0x56041a473a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.637+0000 7f0429122740 4 client.15456 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T02:07:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.637+0000 7f0429122740 1 -- 172.21.15.103:0/1196730428 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56041a35cb80 con 0x56041a471640 2024-09-16T02:07:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.637+0000 7f041ea00640 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x56041a471640 0x56041a473a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.637+0000 7f041ea00640 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x56041a471640 0x56041a473a60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a20760809f3975 server_cookie=f43693bc0563b4fb in_seq=0 out_seq=0 2024-09-16T02:07:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.637+0000 7f0409600640 10 client.15456 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0409600640 1 -- 172.21.15.103:0/1196730428 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56041a35cb80 con 0x56041a471640 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0409600640 10 client.15456 handle_command_reply: tid=0 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0429122740 1 client.15456 shutdown 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0429122740 1 -- 172.21.15.103:0/1196730428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 msgr2=0x56041a3ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0429122740 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 0x56041a3ff180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x56041a3f1400 tx=0x7f04100046c0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0429122740 1 -- 172.21.15.103:0/1196730428 shutdown_connections 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0429122740 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x56041a471640 0x56041a473a60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0429122740 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56041a3f5a60 0x56041a400680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0429122740 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56041a3f09d0 0x56041a3ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0429122740 1 --2- 172.21.15.103:0/1196730428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56041a3f0090 0x56041a3ff180 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0409600640 0 client.15456 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.658+0000 7f0429122740 1 -- 172.21.15.103:0/1196730428 >> 172.21.15.103:0/1196730428 conn(0x56041a3eb700 msgr2=0x56041a3ea060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.659+0000 7f0429122740 1 -- 172.21.15.103:0/1196730428 shutdown_connections 2024-09-16T02:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.659+0000 7f0429122740 1 -- 172.21.15.103:0/1196730428 wait complete. 2024-09-16T02:07:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.659+0000 7f0429122740 20 client.15456 trim_cache size 0 max 16384 2024-09-16T02:07:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-16T02:07:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-16T02:07:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-16T02:07:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-16T02:07:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.659+0000 7f0426600640 1 -- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f04040a3370 msgr2=0x7f04040a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.659+0000 7f0426600640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f04040a3370 0x7f04040a5830 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56041a3ec580 tx=0x7f0410005f90 comp rx=0 tx=0).stop 2024-09-16T02:07:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.659+0000 7f0426600640 1 -- 172.21.15.103:0/3418748508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0420104f50 msgr2=0x7f042019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.659+0000 7f0426600640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0420104f50 0x7f042019a4a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e9f0 tx=0x7f040c00edf0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.660+0000 7f0426600640 1 -- 172.21.15.103:0/3418748508 shutdown_connections 2024-09-16T02:07:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.660+0000 7f0426600640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f04040a3370 0x7f04040a5830 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.660+0000 7f0426600640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0420105910 0x7f04201a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.660+0000 7f0426600640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0420104f50 0x7f042019a4a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.660+0000 7f0426600640 1 --2- 172.21.15.103:0/3418748508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0420103d60 0x7f0420199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.660+0000 7f0426600640 1 -- 172.21.15.103:0/3418748508 >> 172.21.15.103:0/3418748508 conn(0x7f04200ff4e0 msgr2=0x7f04201081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.660+0000 7f0426600640 1 -- 172.21.15.103:0/3418748508 shutdown_connections 2024-09-16T02:07:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-16T02:07:13.660+0000 7f0426600640 1 -- 172.21.15.103:0/3418748508 wait complete. 2024-09-16T02:07:13.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:13 smithi103 ceph-mon[30398]: pgmap v393: 97 pgs: 97 active+clean; 266 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 871 KiB/s wr, 830 op/s 2024-09-16T02:07:13.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:13 smithi136 ceph-mon[35323]: pgmap v393: 97 pgs: 97 active+clean; 266 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 871 KiB/s wr, 830 op/s 2024-09-16T02:07:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:13 smithi172 ceph-mon[35340]: pgmap v393: 97 pgs: 97 active+clean; 266 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 871 KiB/s wr, 830 op/s 2024-09-16T02:07:14.067 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-09-16T02:07:14.068 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-16T02:07:14.069 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-09-16T02:07:14.069 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-16T02:07:14.069 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-09-16T02:07:14.069 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-16T02:07:14.069 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-09-16T02:07:14.069 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-16T02:07:14.069 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-09-16T02:07:14.069 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-16T02:07:14.070 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-09-16T02:07:14.071 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-16T02:07:14.072 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-16T02:07:14.073 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-09-16T02:07:14.074 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-16T02:07:14.074 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-09-16T02:07:14.074 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-16T02:07:14.074 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-09-16T02:07:14.074 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-16T02:07:14.074 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-09-16T02:07:14.074 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-16T02:07:14.074 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-09-16T02:07:14.205 INFO:tasks.cephfs.filesystem:scrub status for tag:e01da340-43e7-4c63-9fd3-791d885d358a - {'path': '/', 'tag': 'e01da340-43e7-4c63-9fd3-791d885d358a', 'options': 'recursive,force'} 2024-09-16T02:07:14.588 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-16T02:07:14.588 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-09-16T02:07:14.588 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-16T02:07:14.588 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-09-16T02:07:14.588 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-16T02:07:14.588 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-09-16T02:07:14.588 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-16T02:07:14.589 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-09-16T02:07:14.590 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-09-16T02:07:14.591 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-16T02:07:14.592 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-09-16T02:07:14.593 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-16T02:07:14.594 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-09-16T02:07:14.950 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-16T02:07:14.950 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-09-16T02:07:14.950 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-16T02:07:14.950 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-09-16T02:07:14.950 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-16T02:07:14.950 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-16T02:07:14.951 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-09-16T02:07:14.952 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-09-16T02:07:14.953 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-16T02:07:14.954 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-09-16T02:07:14.956 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-16T02:07:14.956 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-09-16T02:07:14.956 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-16T02:07:14.956 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-09-16T02:07:14.957 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-16T02:07:14.958 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-09-16T02:07:14.959 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-16T02:07:14.959 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-09-16T02:07:14.959 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-16T02:07:15.557 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-09-16T02:07:15.557 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-16T02:07:15.557 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-09-16T02:07:15.557 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-16T02:07:15.557 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-09-16T02:07:15.557 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-16T02:07:15.557 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-09-16T02:07:15.557 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-09-16T02:07:15.558 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-09-16T02:07:15.559 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-16T02:07:15.560 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-16T02:07:15.561 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-09-16T02:07:15.562 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-16T02:07:15.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-09-16T02:07:15.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-16T02:07:15.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-09-16T02:07:15.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-16T02:07:15.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-09-16T02:07:15.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-16T02:07:15.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-09-16T02:07:15.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-16T02:07:15.563 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-09-16T02:07:16.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:15 smithi103 ceph-mon[30398]: pgmap v394: 97 pgs: 97 active+clean; 242 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 1.3 MiB/s wr, 944 op/s 2024-09-16T02:07:16.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:15 smithi103 ceph-mon[30398]: scrub summary: idle 2024-09-16T02:07:16.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:15 smithi136 ceph-mon[35323]: pgmap v394: 97 pgs: 97 active+clean; 242 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 1.3 MiB/s wr, 944 op/s 2024-09-16T02:07:16.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:15 smithi136 ceph-mon[35323]: scrub summary: idle 2024-09-16T02:07:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:15 smithi172 ceph-mon[35340]: pgmap v394: 97 pgs: 97 active+clean; 242 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 1.3 MiB/s wr, 944 op/s 2024-09-16T02:07:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:15 smithi172 ceph-mon[35340]: scrub summary: idle 2024-09-16T02:07:16.385 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-16T02:07:16.385 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-09-16T02:07:16.385 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-16T02:07:16.385 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-16T02:07:16.386 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-09-16T02:07:16.387 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-16T02:07:16.388 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-09-16T02:07:16.389 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-16T02:07:16.390 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-09-16T02:07:16.391 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-16T02:07:16.391 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-09-16T02:07:16.391 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-16T02:07:16.391 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-09-16T02:07:16.391 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-16T02:07:16.391 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-09-16T02:07:16.391 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-16T02:07:16.391 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-09-16T02:07:16.391 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-16T02:07:16.787 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-09-16T02:07:16.788 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-16T02:07:16.788 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-09-16T02:07:16.788 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-16T02:07:16.788 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-09-16T02:07:16.788 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-16T02:07:16.788 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-09-16T02:07:16.788 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-16T02:07:16.788 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-09-16T02:07:16.788 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-16T02:07:16.788 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-09-16T02:07:16.789 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-16T02:07:16.789 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-09-16T02:07:16.789 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-16T02:07:16.789 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-09-16T02:07:16.789 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-16T02:07:16.789 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-09-16T02:07:16.789 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-16T02:07:16.789 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-09-16T02:07:16.789 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-16T02:07:16.789 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-09-16T02:07:16.792 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-09-16T02:07:16.793 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-16T02:07:16.794 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-09-16T02:07:16.794 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-16T02:07:16.794 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-09-16T02:07:16.794 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-16T02:07:16.794 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-09-16T02:07:16.794 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-16T02:07:16.794 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-09-16T02:07:16.794 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-09-16T02:07:16.795 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-16T02:07:16.796 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-09-16T02:07:16.797 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-16T02:07:17.165 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-16T02:07:17.166 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-16T02:07:17.167 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-09-16T02:07:17.168 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-09-16T02:07:17.169 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-09-16T02:07:17.170 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-16T02:07:17.171 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-09-16T02:07:17.827 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:17 smithi103 ceph-mon[30398]: pgmap v395: 97 pgs: 97 active+clean; 217 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s rd, 1.3 MiB/s wr, 397 op/s 2024-09-16T02:07:17.904 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-16T02:07:17.904 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-09-16T02:07:17.904 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-16T02:07:17.904 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-09-16T02:07:17.904 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-16T02:07:17.904 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-09-16T02:07:17.904 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-16T02:07:17.904 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-09-16T02:07:17.904 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-16T02:07:17.904 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-09-16T02:07:17.905 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-16T02:07:17.905 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-09-16T02:07:17.905 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-16T02:07:17.905 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-09-16T02:07:17.905 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-16T02:07:17.905 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-09-16T02:07:17.905 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-16T02:07:17.905 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-09-16T02:07:17.905 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-16T02:07:17.905 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-09-16T02:07:17.906 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-09-16T02:07:17.907 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-16T02:07:17.908 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-09-16T02:07:17.908 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-16T02:07:17.908 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-09-16T02:07:17.908 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-16T02:07:17.908 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-09-16T02:07:17.908 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-16T02:07:17.908 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-09-16T02:07:17.908 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-16T02:07:17.909 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-09-16T02:07:17.910 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-16T02:07:18.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:17 smithi136 ceph-mon[35323]: pgmap v395: 97 pgs: 97 active+clean; 217 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s rd, 1.3 MiB/s wr, 397 op/s 2024-09-16T02:07:18.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:17 smithi172 ceph-mon[35340]: pgmap v395: 97 pgs: 97 active+clean; 217 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s rd, 1.3 MiB/s wr, 397 op/s 2024-09-16T02:07:18.656 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-09-16T02:07:18.656 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-16T02:07:18.657 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-09-16T02:07:18.658 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-16T02:07:18.659 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-16T02:07:18.660 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-09-16T02:07:18.661 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-16T02:07:18.662 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-09-16T02:07:19.029 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-16T02:07:19.029 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-09-16T02:07:19.030 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-16T02:07:19.030 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-09-16T02:07:19.030 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-16T02:07:19.030 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-09-16T02:07:19.030 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-16T02:07:19.030 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-09-16T02:07:19.030 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-16T02:07:19.030 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-09-16T02:07:19.031 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-16T02:07:19.031 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-09-16T02:07:19.031 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-16T02:07:19.031 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-09-16T02:07:19.031 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-16T02:07:19.031 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-09-16T02:07:19.031 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-16T02:07:19.031 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-09-16T02:07:19.031 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-16T02:07:19.032 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-09-16T02:07:19.032 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-16T02:07:19.032 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-09-16T02:07:19.032 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-16T02:07:19.032 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-09-16T02:07:19.032 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-16T02:07:19.032 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-09-16T02:07:19.032 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-16T02:07:19.033 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-09-16T02:07:19.033 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-16T02:07:19.033 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-09-16T02:07:19.033 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-16T02:07:19.033 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-09-16T02:07:19.033 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-16T02:07:19.033 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-09-16T02:07:19.033 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-16T02:07:19.034 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-09-16T02:07:19.036 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-16T02:07:19.036 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-09-16T02:07:19.037 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-16T02:07:19.037 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-09-16T02:07:19.037 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-16T02:07:19.037 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-09-16T02:07:19.037 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-16T02:07:19.037 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-09-16T02:07:19.037 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-16T02:07:19.038 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-09-16T02:07:19.038 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-16T02:07:19.038 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-09-16T02:07:19.038 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-16T02:07:19.038 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-09-16T02:07:19.038 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-16T02:07:19.038 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-09-16T02:07:19.038 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-16T02:07:19.038 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-09-16T02:07:19.038 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-09-16T02:07:19.039 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-09-16T02:07:19.040 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-16T02:07:19.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-09-16T02:07:19.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-16T02:07:19.531 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-09-16T02:07:19.532 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-09-16T02:07:19.533 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-09-16T02:07:19.534 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-16T02:07:19.535 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-09-16T02:07:19.536 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-16T02:07:19.537 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-09-16T02:07:19.537 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-16T02:07:19.537 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-09-16T02:07:19.537 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-16T02:07:19.537 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-09-16T02:07:19.537 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-16T02:07:19.537 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-09-16T02:07:19.537 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-16T02:07:19.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:19 smithi103 ceph-mon[30398]: pgmap v396: 97 pgs: 97 active+clean; 213 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s rd, 1.7 MiB/s wr, 494 op/s 2024-09-16T02:07:20.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:19 smithi136 ceph-mon[35323]: pgmap v396: 97 pgs: 97 active+clean; 213 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s rd, 1.7 MiB/s wr, 494 op/s 2024-09-16T02:07:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:19 smithi172 ceph-mon[35340]: pgmap v396: 97 pgs: 97 active+clean; 213 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s rd, 1.7 MiB/s wr, 494 op/s 2024-09-16T02:07:20.277 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-09-16T02:07:20.277 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-16T02:07:20.277 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-09-16T02:07:20.277 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-16T02:07:20.277 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-09-16T02:07:20.277 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-16T02:07:20.277 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-16T02:07:20.278 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-09-16T02:07:20.279 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-16T02:07:20.280 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-16T02:07:20.281 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-09-16T02:07:20.282 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-16T02:07:20.283 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-09-16T02:07:20.283 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-16T02:07:20.283 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-09-16T02:07:20.283 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-16T02:07:20.283 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-09-16T02:07:20.283 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-16T02:07:20.283 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-09-16T02:07:20.283 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-16T02:07:20.283 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-09-16T02:07:20.866 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-16T02:07:20.866 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-09-16T02:07:20.866 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-16T02:07:20.866 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-09-16T02:07:20.866 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-09-16T02:07:20.867 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-09-16T02:07:20.868 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-16T02:07:20.869 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-16T02:07:20.870 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-09-16T02:07:20.871 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-16T02:07:20.872 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-09-16T02:07:20.872 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-16T02:07:20.872 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-09-16T02:07:20.872 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-16T02:07:20.872 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-09-16T02:07:20.872 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-16T02:07:21.242 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-09-16T02:07:21.243 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-16T02:07:21.243 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-09-16T02:07:21.243 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-16T02:07:21.243 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-09-16T02:07:21.243 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-16T02:07:21.243 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-09-16T02:07:21.243 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-16T02:07:21.243 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-09-16T02:07:21.244 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-16T02:07:21.244 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-09-16T02:07:21.244 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-09-16T02:07:21.245 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-09-16T02:07:21.246 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-09-16T02:07:21.247 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-16T02:07:21.248 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-09-16T02:07:21.248 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-16T02:07:21.248 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-09-16T02:07:21.248 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-16T02:07:21.248 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-09-16T02:07:21.248 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-16T02:07:21.248 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-09-16T02:07:21.248 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-16T02:07:21.248 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-09-16T02:07:21.248 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-16T02:07:21.249 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-09-16T02:07:21.249 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-16T02:07:21.249 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-09-16T02:07:21.249 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-16T02:07:21.249 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-09-16T02:07:21.249 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-16T02:07:21.249 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-09-16T02:07:21.249 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-16T02:07:21.249 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-09-16T02:07:21.249 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-16T02:07:21.250 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-09-16T02:07:21.250 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-16T02:07:21.250 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-09-16T02:07:21.250 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-16T02:07:21.250 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-09-16T02:07:21.250 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-16T02:07:21.250 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-09-16T02:07:21.250 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-16T02:07:21.250 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-09-16T02:07:21.817 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-16T02:07:21.818 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-09-16T02:07:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-16T02:07:21.820 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-16T02:07:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-16T02:07:21.822 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-09-16T02:07:21.823 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-16T02:07:21.824 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-09-16T02:07:21.824 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-16T02:07:22.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:21 smithi103 ceph-mon[30398]: pgmap v397: 97 pgs: 97 active+clean; 273 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-09-16T02:07:22.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:21 smithi136 ceph-mon[35323]: pgmap v397: 97 pgs: 97 active+clean; 273 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-09-16T02:07:22.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:21 smithi172 ceph-mon[35340]: pgmap v397: 97 pgs: 97 active+clean; 273 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-09-16T02:07:22.604 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-09-16T02:07:22.604 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-16T02:07:22.604 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-09-16T02:07:22.604 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-16T02:07:22.604 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-09-16T02:07:22.604 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-16T02:07:22.604 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-09-16T02:07:22.604 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-16T02:07:22.604 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-09-16T02:07:22.605 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-16T02:07:22.606 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-09-16T02:07:22.607 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-09-16T02:07:22.608 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-16T02:07:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-09-16T02:07:22.610 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-16T02:07:22.610 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-09-16T02:07:22.610 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-16T02:07:22.610 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-09-16T02:07:22.610 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-16T02:07:23.041 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-09-16T02:07:23.041 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-16T02:07:23.041 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-09-16T02:07:23.041 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-16T02:07:23.041 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-09-16T02:07:23.041 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-16T02:07:23.041 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-09-16T02:07:23.041 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-16T02:07:23.041 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-09-16T02:07:23.042 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-09-16T02:07:23.043 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-16T02:07:23.044 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-09-16T02:07:23.044 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-16T02:07:23.044 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-09-16T02:07:23.044 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-16T02:07:23.044 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-09-16T02:07:23.045 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-16T02:07:23.046 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-09-16T02:07:23.047 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-16T02:07:23.048 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-09-16T02:07:23.048 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-16T02:07:23.048 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-09-16T02:07:23.048 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-16T02:07:23.048 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-09-16T02:07:23.423 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-16T02:07:23.423 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-09-16T02:07:23.423 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-16T02:07:23.423 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-09-16T02:07:23.424 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-09-16T02:07:23.425 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-09-16T02:07:23.426 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-09-16T02:07:23.427 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-09-16T02:07:23.428 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-16T02:07:23.429 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-09-16T02:07:23.429 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-16T02:07:23.429 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-09-16T02:07:23.429 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-16T02:07:23.429 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-09-16T02:07:23.429 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-16T02:07:23.429 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-09-16T02:07:23.429 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-16T02:07:23.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-09-16T02:07:23.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-16T02:07:23.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-09-16T02:07:23.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-16T02:07:23.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-09-16T02:07:23.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-16T02:07:23.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-09-16T02:07:23.492 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-16T02:07:23.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-09-16T02:07:23.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-16T02:07:23.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-09-16T02:07:23.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-16T02:07:23.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-09-16T02:07:23.493 INFO:tasks.workunit.client.0.smithi103.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-16T02:07:23.493 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T02:07:23.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T02:07:23.558 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-09-16T02:07:23.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T02:07:24.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:23 smithi103 ceph-mon[30398]: pgmap v398: 97 pgs: 97 active+clean; 279 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 130 MiB/s rd, 1.2 MiB/s wr, 430 op/s 2024-09-16T02:07:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:23 smithi136 ceph-mon[35323]: pgmap v398: 97 pgs: 97 active+clean; 279 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 130 MiB/s rd, 1.2 MiB/s wr, 430 op/s 2024-09-16T02:07:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:23 smithi172 ceph-mon[35340]: pgmap v398: 97 pgs: 97 active+clean; 279 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 130 MiB/s rd, 1.2 MiB/s wr, 430 op/s 2024-09-16T02:07:24.642 DEBUG:teuthology.parallel:result is None 2024-09-16T02:07:24.643 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T02:07:24.710 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T02:07:24.710 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-16T02:07:24.720 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi103.front.sepia.ceph.com 2024-09-16T02:07:24.732 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-16T02:07:24.787 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-16T02:07:26.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:25 smithi103 ceph-mon[30398]: pgmap v399: 97 pgs: 97 active+clean; 183 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 136 MiB/s rd, 1.3 MiB/s wr, 468 op/s 2024-09-16T02:07:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:25 smithi136 ceph-mon[35323]: pgmap v399: 97 pgs: 97 active+clean; 183 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 136 MiB/s rd, 1.3 MiB/s wr, 468 op/s 2024-09-16T02:07:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:25 smithi172 ceph-mon[35340]: pgmap v399: 97 pgs: 97 active+clean; 183 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 136 MiB/s rd, 1.3 MiB/s wr, 468 op/s 2024-09-16T02:07:28.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:27 smithi103 ceph-mon[30398]: pgmap v400: 97 pgs: 97 active+clean; 129 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s rd, 1.2 MiB/s wr, 437 op/s 2024-09-16T02:07:28.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:27 smithi136 ceph-mon[35323]: pgmap v400: 97 pgs: 97 active+clean; 129 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s rd, 1.2 MiB/s wr, 437 op/s 2024-09-16T02:07:28.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:27 smithi172 ceph-mon[35340]: pgmap v400: 97 pgs: 97 active+clean; 129 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s rd, 1.2 MiB/s wr, 437 op/s 2024-09-16T02:07:30.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:29 smithi103 ceph-mon[30398]: pgmap v401: 97 pgs: 97 active+clean; 101 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s rd, 1.1 MiB/s wr, 379 op/s 2024-09-16T02:07:30.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:29 smithi136 ceph-mon[35323]: pgmap v401: 97 pgs: 97 active+clean; 101 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s rd, 1.1 MiB/s wr, 379 op/s 2024-09-16T02:07:30.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:29 smithi172 ceph-mon[35340]: pgmap v401: 97 pgs: 97 active+clean; 101 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s rd, 1.1 MiB/s wr, 379 op/s 2024-09-16T02:07:32.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:31 smithi103 ceph-mon[30398]: pgmap v402: 97 pgs: 97 active+clean; 8.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 738 KiB/s wr, 285 op/s 2024-09-16T02:07:32.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:31 smithi136 ceph-mon[35323]: pgmap v402: 97 pgs: 97 active+clean; 8.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 738 KiB/s wr, 285 op/s 2024-09-16T02:07:32.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:31 smithi172 ceph-mon[35340]: pgmap v402: 97 pgs: 97 active+clean; 8.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 738 KiB/s wr, 285 op/s 2024-09-16T02:07:34.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:33 smithi103 ceph-mon[30398]: pgmap v403: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 663 KiB/s wr, 243 op/s 2024-09-16T02:07:34.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:33 smithi136 ceph-mon[35323]: pgmap v403: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 663 KiB/s wr, 243 op/s 2024-09-16T02:07:34.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:33 smithi172 ceph-mon[35340]: pgmap v403: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 663 KiB/s wr, 243 op/s 2024-09-16T02:07:36.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:35 smithi103 ceph-mon[30398]: pgmap v404: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 428 KiB/s wr, 171 op/s 2024-09-16T02:07:36.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:35 smithi136 ceph-mon[35323]: pgmap v404: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 428 KiB/s wr, 171 op/s 2024-09-16T02:07:36.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:35 smithi172 ceph-mon[35340]: pgmap v404: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 428 KiB/s wr, 171 op/s 2024-09-16T02:07:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:37 smithi103 ceph-mon[30398]: pgmap v405: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 320 KiB/s wr, 105 op/s 2024-09-16T02:07:38.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:37 smithi136 ceph-mon[35323]: pgmap v405: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 320 KiB/s wr, 105 op/s 2024-09-16T02:07:38.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:37 smithi172 ceph-mon[35340]: pgmap v405: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 320 KiB/s wr, 105 op/s 2024-09-16T02:07:40.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:39 smithi103 ceph-mon[30398]: pgmap v406: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-09-16T02:07:40.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:39 smithi136 ceph-mon[35323]: pgmap v406: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-09-16T02:07:40.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:39 smithi172 ceph-mon[35340]: pgmap v406: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-09-16T02:07:42.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:41 smithi103 ceph-mon[30398]: pgmap v407: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-16T02:07:42.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:41 smithi136 ceph-mon[35323]: pgmap v407: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-16T02:07:42.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:41 smithi172 ceph-mon[35340]: pgmap v407: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-16T02:07:44.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:43 smithi103 ceph-mon[30398]: pgmap v408: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-16T02:07:44.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:43 smithi136 ceph-mon[35323]: pgmap v408: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-16T02:07:44.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:43 smithi172 ceph-mon[35340]: pgmap v408: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-16T02:07:44.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T02:07:44.589 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.100+0000 7f438cc00640 1 -- 172.21.15.103:0/1382394922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388077040 msgr2=0x7f4388112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.100+0000 7f438cc00640 1 --2- 172.21.15.103:0/1382394922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388077040 0x7f4388112330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4374009900 tx=0x7f437402a400 comp rx=0 tx=0).stop 2024-09-16T02:07:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.100+0000 7f438cc00640 1 -- 172.21.15.103:0/1382394922 shutdown_connections 2024-09-16T02:07:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.100+0000 7f438cc00640 1 --2- 172.21.15.103:0/1382394922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4388112870 0x7f4388114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.100+0000 7f438cc00640 1 --2- 172.21.15.103:0/1382394922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388077040 0x7f4388112330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.100+0000 7f438cc00640 1 --2- 172.21.15.103:0/1382394922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4388076720 0x7f4388076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.100+0000 7f438cc00640 1 -- 172.21.15.103:0/1382394922 >> 172.21.15.103:0/1382394922 conn(0x7f43880ff8d0 msgr2=0x7f4388101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.100+0000 7f438cc00640 1 -- 172.21.15.103:0/1382394922 shutdown_connections 2024-09-16T02:07:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.100+0000 7f438cc00640 1 -- 172.21.15.103:0/1382394922 wait complete. 2024-09-16T02:07:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.101+0000 7f438cc00640 1 Processor -- start 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.101+0000 7f438cc00640 1 -- start start 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f438cc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4388076720 0x7f43881a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f438cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4388077040 0x7f43881a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f438cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388112870 0x7f438819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f438cc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43881183e0 con 0x7f4388076720 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f438cc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4388118260 con 0x7f4388112870 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f438cc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4388118560 con 0x7f4388077040 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f4387400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4388076720 0x7f43881a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f4387400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4388076720 0x7f43881a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56134/0 (socket says 172.21.15.103:56134) 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f4387400640 1 -- 172.21.15.103:0/1991527343 learned_addr learned my addr 172.21.15.103:0/1991527343 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f4386a00640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4388077040 0x7f43881a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.102+0000 7f4387e00640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388112870 0x7f438819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f4387400640 1 -- 172.21.15.103:0/1991527343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4388077040 msgr2=0x7f43881a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f4387400640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4388077040 0x7f43881a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f4387400640 1 -- 172.21.15.103:0/1991527343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388112870 msgr2=0x7f438819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f4387400640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388112870 0x7f438819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f4387400640 1 -- 172.21.15.103:0/1991527343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438819ef60 con 0x7f4388076720 2024-09-16T02:07:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f4387400640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4388076720 0x7f43881a4620 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f437800ea60 tx=0x7f437800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f437fe00640 1 -- 172.21.15.103:0/1991527343 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437800cde0 con 0x7f4388076720 2024-09-16T02:07:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f438cc00640 1 -- 172.21.15.103:0/1991527343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438819f250 con 0x7f4388076720 2024-09-16T02:07:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f437fe00640 1 -- 172.21.15.103:0/1991527343 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43780035f0 con 0x7f4388076720 2024-09-16T02:07:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f438cc00640 1 -- 172.21.15.103:0/1991527343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43881ab5a0 con 0x7f4388076720 2024-09-16T02:07:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.103+0000 7f437fe00640 1 -- 172.21.15.103:0/1991527343 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378005220 con 0x7f4388076720 2024-09-16T02:07:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.106+0000 7f4390bd2740 1 Processor -- start 2024-09-16T02:07:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.106+0000 7f437fe00640 1 -- 172.21.15.103:0/1991527343 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f43780037a0 con 0x7f4388076720 2024-09-16T02:07:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.106+0000 7f4390bd2740 1 -- start start 2024-09-16T02:07:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c5d50 0x5587e44c6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44d0f30 0x5587e44d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4390bd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5587e442c3e0 con 0x5587e44c6670 2024-09-16T02:07:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4390bd2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5587e43d3d80 con 0x5587e44d0f30 2024-09-16T02:07:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4390bd2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5587e44283a0 con 0x5587e44c5d50 2024-09-16T02:07:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4387400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4387400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56138/0 (socket says 172.21.15.103:56138) 2024-09-16T02:07:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4387400640 1 -- 172.21.15.103:0/822109995 learned_addr learned my addr 172.21.15.103:0/822109995 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4387e00640 1 --2- 172.21.15.103:0/822109995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c5d50 0x5587e44c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.107+0000 7f4386a00640 1 --2- 172.21.15.103:0/822109995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44d0f30 0x5587e44d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f4387e00640 1 -- 172.21.15.103:0/822109995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44d0f30 msgr2=0x5587e44d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f4387e00640 1 --2- 172.21.15.103:0/822109995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44d0f30 0x5587e44d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f4387e00640 1 -- 172.21.15.103:0/822109995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 msgr2=0x5587e44d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f4387e00640 1 --2- 172.21.15.103:0/822109995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f4387e00640 1 -- 172.21.15.103:0/822109995 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e43f2ae0 con 0x5587e44c5d50 2024-09-16T02:07:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f437fe00640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f436c0a3240 0x7f436c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f4387e00640 1 --2- 172.21.15.103:0/822109995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c5d50 0x5587e44c6130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f438807aa10 tx=0x7f4380010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa7319e4ab882594 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f437fe00640 1 -- 172.21.15.103:0/1991527343 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f43780cfbe0 con 0x7f4388076720 2024-09-16T02:07:45.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f4387400640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f436c0a3240 0x7f436c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f4373e00640 1 -- 172.21.15.103:0/822109995 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380016070 con 0x5587e44c5d50 2024-09-16T02:07:45.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f4373e00640 1 -- 172.21.15.103:0/822109995 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4380004d20 con 0x5587e44c5d50 2024-09-16T02:07:45.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.108+0000 7f4373e00640 1 -- 172.21.15.103:0/822109995 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380012810 con 0x5587e44c5d50 2024-09-16T02:07:45.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.109+0000 7f4390bd2740 1 -- 172.21.15.103:0/822109995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c5d50 msgr2=0x5587e44c6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:45.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.109+0000 7f4387400640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f436c0a3240 0x7f436c0a5700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f437800c760 tx=0x7f43780d3620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:45.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.109+0000 7f4390bd2740 1 --2- 172.21.15.103:0/822109995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c5d50 0x5587e44c6130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f438807aa10 tx=0x7f4380010740 comp rx=0 tx=0).stop 2024-09-16T02:07:45.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.109+0000 7f4390bd2740 1 -- 172.21.15.103:0/822109995 shutdown_connections 2024-09-16T02:07:45.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.109+0000 7f4390bd2740 1 --2- 172.21.15.103:0/822109995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44d0f30 0x5587e44d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.109+0000 7f4390bd2740 1 --2- 172.21.15.103:0/822109995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.109+0000 7f4390bd2740 1 --2- 172.21.15.103:0/822109995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c5d50 0x5587e44c6130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.109+0000 7f4390bd2740 1 -- 172.21.15.103:0/822109995 >> 172.21.15.103:0/822109995 conn(0x5587e44c22e0 msgr2=0x5587e44c26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:45.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.109+0000 7f4390bd2740 1 -- 172.21.15.103:0/822109995 shutdown_connections 2024-09-16T02:07:45.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.110+0000 7f4390bd2740 1 -- 172.21.15.103:0/822109995 wait complete. 2024-09-16T02:07:45.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.110+0000 7f4390bd2740 1 -- start start 2024-09-16T02:07:45.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.110+0000 7f4390bd2740 1 Processor -- start 2024-09-16T02:07:45.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.110+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c5d50 0x5587e44cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c6670 0x5587e44ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44d0f30 0x5587e44ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4390bd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5587e44d3ba0 con 0x5587e44c5d50 2024-09-16T02:07:45.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4390bd2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5587e43e70f0 con 0x5587e44c6670 2024-09-16T02:07:45.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4390bd2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5587e44d3d20 con 0x5587e44d0f30 2024-09-16T02:07:45.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4390bd2740 10 client.0 _ll_register_callbacks cb 0x5587e43d89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:45.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4387e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c5d50 0x5587e44cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4387e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c5d50 0x5587e44cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56140/0 (socket says 172.21.15.103:56140) 2024-09-16T02:07:45.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4387e00640 1 -- 172.21.15.103:0/157319604 learned_addr learned my addr 172.21.15.103:0/157319604 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:45.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4386a00640 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c6670 0x5587e44ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4387e00640 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44d0f30 0x5587e44ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4386a00640 1 -- 172.21.15.103:0/157319604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44d0f30 msgr2=0x5587e44ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4386a00640 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44d0f30 0x5587e44ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4386a00640 1 -- 172.21.15.103:0/157319604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c5d50 msgr2=0x5587e44cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4386a00640 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c5d50 0x5587e44cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.111+0000 7f4386a00640 1 -- 172.21.15.103:0/157319604 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e419d3c0 con 0x5587e44c6670 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.112+0000 7f4386a00640 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c6670 0x5587e44ce420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4374005940 tx=0x7f43740317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42d80d88e1620e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.112+0000 7f4370c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.112+0000 7f4370c00640 1 -- 172.21.15.103:0/157319604 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374035040 con 0x5587e44c6670 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.112+0000 7f4370c00640 1 -- 172.21.15.103:0/157319604 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f4374004240 con 0x5587e44c6670 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.112+0000 7f4390bd2740 1 -- 172.21.15.103:0/157319604 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4380004d20 con 0x5587e44c6670 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.112+0000 7f4370c00640 1 -- 172.21.15.103:0/157319604 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f437404b030 con 0x5587e44c6670 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.113+0000 7f4390bd2740 10 client.25445 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.113+0000 7f4390bd2740 1 -- 172.21.15.103:0/157319604 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5587e44df910 con 0x5587e44c6670 2024-09-16T02:07:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.113+0000 7f4370c00640 1 -- 172.21.15.103:0/157319604 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4380004d20 con 0x5587e44c6670 2024-09-16T02:07:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.113+0000 7f4370c00640 1 -- 172.21.15.103:0/157319604 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4374046510 con 0x5587e44c6670 2024-09-16T02:07:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.113+0000 7f4390bd2740 10 client.25445 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.113+0000 7f4390bd2740 10 client.25445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.113+0000 7f4390bd2740 20 client.25445 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.113+0000 7f4390bd2740 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5587e44e0dd0 0x5587e44e31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.113+0000 7f4390bd2740 4 client.25445 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.113+0000 7f4390bd2740 1 -- 172.21.15.103:0/157319604 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587e4439950 con 0x5587e44e0dd0 2024-09-16T02:07:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.114+0000 7f4387e00640 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5587e44e0dd0 0x5587e44e31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.114+0000 7f4387e00640 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5587e44e0dd0 0x5587e44e31c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc18903d9121a16 server_cookie=15631fa5e7d96f93 in_seq=0 out_seq=0 2024-09-16T02:07:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.114+0000 7f4370c00640 10 client.25445 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4370c00640 1 -- 172.21.15.103:0/157319604 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587e4439950 con 0x5587e44e0dd0 2024-09-16T02:07:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4370c00640 10 client.25445 handle_command_reply: tid=0 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 client.25445 shutdown 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 -- 172.21.15.103:0/157319604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c6670 msgr2=0x5587e44ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c6670 0x5587e44ce420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4374005940 tx=0x7f43740317e0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 -- 172.21.15.103:0/157319604 shutdown_connections 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5587e44e0dd0 0x5587e44e31c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44d0f30 0x5587e44ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c6670 0x5587e44ce420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 --2- 172.21.15.103:0/157319604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c5d50 0x5587e44cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4370c00640 0 client.25445 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 -- 172.21.15.103:0/157319604 >> 172.21.15.103:0/157319604 conn(0x5587e44c22e0 msgr2=0x5587e44c8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 -- 172.21.15.103:0/157319604 shutdown_connections 2024-09-16T02:07:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 1 -- 172.21.15.103:0/157319604 wait complete. 2024-09-16T02:07:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.115+0000 7f4390bd2740 20 client.25445 trim_cache size 0 max 16384 2024-09-16T02:07:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.133+0000 7f4390bd2740 1 Processor -- start 2024-09-16T02:07:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.133+0000 7f4390bd2740 1 -- start start 2024-09-16T02:07:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.133+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 0x5587e44e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44e8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44d0f30 0x5587e44cb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4390bd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43881b65a0 con 0x5587e44c6670 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4390bd2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43881b3e70 con 0x5587e44c5d50 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4390bd2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43881b1cc0 con 0x5587e44d0f30 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4386a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44e8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4386a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44e8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56144/0 (socket says 172.21.15.103:56144) 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4386a00640 1 -- 172.21.15.103:0/1099429638 learned_addr learned my addr 172.21.15.103:0/1099429638 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4387e00640 1 --2- 172.21.15.103:0/1099429638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 0x5587e44e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4387e00640 1 --2- 172.21.15.103:0/1099429638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44d0f30 0x5587e44cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4386a00640 1 -- 172.21.15.103:0/1099429638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44d0f30 msgr2=0x5587e44cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4386a00640 1 --2- 172.21.15.103:0/1099429638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44d0f30 0x5587e44cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4386a00640 1 -- 172.21.15.103:0/1099429638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 msgr2=0x5587e44e8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4386a00640 1 --2- 172.21.15.103:0/1099429638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 0x5587e44e8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4386a00640 1 -- 172.21.15.103:0/1099429638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e44c2f70 con 0x5587e44c6670 2024-09-16T02:07:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.134+0000 7f4387e00640 1 --2- 172.21.15.103:0/1099429638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 0x5587e44e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4386a00640 1 --2- 172.21.15.103:0/1099429638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44e8fb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4374046930 tx=0x7f4374000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa345c839e7e1cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4373400640 1 -- 172.21.15.103:0/1099429638 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374035070 con 0x5587e44c6670 2024-09-16T02:07:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4373400640 1 -- 172.21.15.103:0/1099429638 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4374046b20 con 0x5587e44c6670 2024-09-16T02:07:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4373400640 1 -- 172.21.15.103:0/1099429638 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43740052d0 con 0x5587e44c6670 2024-09-16T02:07:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4390bd2740 1 -- 172.21.15.103:0/1099429638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 msgr2=0x5587e44e8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4390bd2740 1 --2- 172.21.15.103:0/1099429638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44e8fb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4374046930 tx=0x7f4374000bc0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4390bd2740 1 -- 172.21.15.103:0/1099429638 shutdown_connections 2024-09-16T02:07:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4390bd2740 1 --2- 172.21.15.103:0/1099429638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44d0f30 0x5587e44cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4390bd2740 1 --2- 172.21.15.103:0/1099429638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44c6670 0x5587e44e8fb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4390bd2740 1 --2- 172.21.15.103:0/1099429638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 0x5587e44e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.135+0000 7f4390bd2740 1 -- 172.21.15.103:0/1099429638 >> 172.21.15.103:0/1099429638 conn(0x5587e44c22e0 msgr2=0x5587e44cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.136+0000 7f4390bd2740 1 -- 172.21.15.103:0/1099429638 shutdown_connections 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.136+0000 7f4390bd2740 1 -- 172.21.15.103:0/1099429638 wait complete. 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.136+0000 7f4390bd2740 1 -- start start 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.136+0000 7f4390bd2740 1 Processor -- start 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 0x5587e44d4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c6670 0x5587e44d54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4390bd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44d0f30 0x5587e44d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4390bd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43881b2330 con 0x5587e44d0f30 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4390bd2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43881b37a0 con 0x5587e44c5d50 2024-09-16T02:07:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4390bd2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43881b2a00 con 0x5587e44c6670 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4390bd2740 10 client.0 _ll_register_callbacks cb 0x7f43740095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4387e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44d0f30 0x5587e44d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4387e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44d0f30 0x5587e44d59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56150/0 (socket says 172.21.15.103:56150) 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4387e00640 1 -- 172.21.15.103:0/2782506441 learned_addr learned my addr 172.21.15.103:0/2782506441 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4387e00640 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 0x5587e44d4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4386a00640 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c6670 0x5587e44d54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4387e00640 1 -- 172.21.15.103:0/2782506441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c6670 msgr2=0x5587e44d54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4387e00640 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c6670 0x5587e44d54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4387e00640 1 -- 172.21.15.103:0/2782506441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 msgr2=0x5587e44d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4387e00640 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 0x5587e44d4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.137+0000 7f4387e00640 1 -- 172.21.15.103:0/2782506441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e454ace0 con 0x5587e44d0f30 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.138+0000 7f4387e00640 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44d0f30 0x5587e44d59f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4380009e00 tx=0x7f438001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4ff5596daa7b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.138+0000 7f4370c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:07:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.138+0000 7f4370c00640 1 -- 172.21.15.103:0/2782506441 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380024050 con 0x5587e44d0f30 2024-09-16T02:07:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.138+0000 7f4370c00640 1 -- 172.21.15.103:0/2782506441 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f438002ba30 con 0x5587e44d0f30 2024-09-16T02:07:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.138+0000 7f4390bd2740 1 -- 172.21.15.103:0/2782506441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4374046b20 con 0x5587e44d0f30 2024-09-16T02:07:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.138+0000 7f4370c00640 1 -- 172.21.15.103:0/2782506441 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f438003a050 con 0x5587e44d0f30 2024-09-16T02:07:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.138+0000 7f4390bd2740 10 client.15474 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.138+0000 7f4390bd2740 1 -- 172.21.15.103:0/2782506441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5587e454bfd0 con 0x5587e44d0f30 2024-09-16T02:07:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.139+0000 7f4370c00640 1 -- 172.21.15.103:0/2782506441 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4374046b20 con 0x5587e44d0f30 2024-09-16T02:07:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.139+0000 7f4370c00640 1 -- 172.21.15.103:0/2782506441 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f438002a040 con 0x5587e44d0f30 2024-09-16T02:07:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.139+0000 7f4390bd2740 10 client.15474 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.139+0000 7f4390bd2740 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.139+0000 7f4390bd2740 20 client.15474 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.139+0000 7f4390bd2740 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5587e454c460 0x5587e454e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.139+0000 7f4390bd2740 4 client.15474 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T02:07:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.139+0000 7f4390bd2740 1 -- 172.21.15.103:0/2782506441 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587e4433e60 con 0x5587e454c460 2024-09-16T02:07:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.139+0000 7f4386a00640 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5587e454c460 0x5587e454e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4386a00640 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5587e454c460 0x5587e454e850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa544e258068525 server_cookie=c398f787d0e7086 in_seq=0 out_seq=0 2024-09-16T02:07:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4370c00640 10 client.15474 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4370c00640 1 -- 172.21.15.103:0/2782506441 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5587e4433e60 con 0x5587e454c460 2024-09-16T02:07:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4370c00640 10 client.15474 handle_command_reply: tid=0 2024-09-16T02:07:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4390bd2740 1 client.15474 shutdown 2024-09-16T02:07:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4390bd2740 1 -- 172.21.15.103:0/2782506441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44d0f30 msgr2=0x5587e44d59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4390bd2740 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44d0f30 0x5587e44d59f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4380009e00 tx=0x7f438001d040 comp rx=0 tx=0).stop 2024-09-16T02:07:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4390bd2740 1 -- 172.21.15.103:0/2782506441 shutdown_connections 2024-09-16T02:07:45.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4390bd2740 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x5587e454c460 0x5587e454e850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4390bd2740 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e44d0f30 0x5587e44d59f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4390bd2740 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587e44c6670 0x5587e44d54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4390bd2740 1 --2- 172.21.15.103:0/2782506441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587e44c5d50 0x5587e44d4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4390bd2740 1 -- 172.21.15.103:0/2782506441 >> 172.21.15.103:0/2782506441 conn(0x5587e44c22e0 msgr2=0x5587e44c0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:45.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4370c00640 0 client.15474 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:45.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.140+0000 7f4390bd2740 1 -- 172.21.15.103:0/2782506441 shutdown_connections 2024-09-16T02:07:45.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f4390bd2740 1 -- 172.21.15.103:0/2782506441 wait complete. 2024-09-16T02:07:45.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f4390bd2740 20 client.15474 trim_cache size 0 max 16384 2024-09-16T02:07:45.146 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:07:45.147 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-16T02:07:45.147 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-16T02:07:45.147 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:07:45.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f438cc00640 1 -- 172.21.15.103:0/1991527343 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f436c0a3240 msgr2=0x7f436c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:45.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f438cc00640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f436c0a3240 0x7f436c0a5700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f437800c760 tx=0x7f43780d3620 comp rx=0 tx=0).stop 2024-09-16T02:07:45.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f438cc00640 1 -- 172.21.15.103:0/1991527343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4388076720 msgr2=0x7f43881a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:45.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f438cc00640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4388076720 0x7f43881a4620 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f437800ea60 tx=0x7f437800ef20 comp rx=0 tx=0).stop 2024-09-16T02:07:45.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f438cc00640 1 -- 172.21.15.103:0/1991527343 shutdown_connections 2024-09-16T02:07:45.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f438cc00640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f436c0a3240 0x7f436c0a5700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f438cc00640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388112870 0x7f438819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f438cc00640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4388077040 0x7f43881a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f438cc00640 1 --2- 172.21.15.103:0/1991527343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4388076720 0x7f43881a4620 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:45.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.141+0000 7f438cc00640 1 -- 172.21.15.103:0/1991527343 >> 172.21.15.103:0/1991527343 conn(0x7f43880ff8d0 msgr2=0x7f4388101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:45.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.142+0000 7f438cc00640 1 -- 172.21.15.103:0/1991527343 shutdown_connections 2024-09-16T02:07:45.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:45.142+0000 7f438cc00640 1 -- 172.21.15.103:0/1991527343 wait complete. 2024-09-16T02:07:45.913 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:45 smithi103 ceph-mon[30398]: pgmap v409: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T02:07:45.951 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T02:07:45.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:07:46.025 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:45 smithi136 ceph-mon[35323]: pgmap v409: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T02:07:46.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:45 smithi172 ceph-mon[35340]: pgmap v409: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T02:07:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:46.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.835+0000 7f5022200640 1 -- 172.21.15.103:0/1295241434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c1047e0 msgr2=0x7f501c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:46.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.835+0000 7f5022200640 1 --2- 172.21.15.103:0/1295241434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c1047e0 0x7f501c108830 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5008009900 tx=0x7f500802a410 comp rx=0 tx=0).stop 2024-09-16T02:07:46.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.835+0000 7f5022200640 1 -- 172.21.15.103:0/1295241434 shutdown_connections 2024-09-16T02:07:46.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.835+0000 7f5022200640 1 --2- 172.21.15.103:0/1295241434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f501c108f60 0x7f501c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.835+0000 7f5022200640 1 --2- 172.21.15.103:0/1295241434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c1047e0 0x7f501c108830 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.835+0000 7f5022200640 1 --2- 172.21.15.103:0/1295241434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f501c103e30 0x7f501c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.835+0000 7f5022200640 1 -- 172.21.15.103:0/1295241434 >> 172.21.15.103:0/1295241434 conn(0x7f501c0ff8d0 msgr2=0x7f501c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.835+0000 7f5022200640 1 -- 172.21.15.103:0/1295241434 shutdown_connections 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.835+0000 7f5022200640 1 -- 172.21.15.103:0/1295241434 wait complete. 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.836+0000 7f5022200640 1 Processor -- start 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.836+0000 7f5022200640 1 -- start start 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.836+0000 7f5022200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c103e30 0x7f501c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.836+0000 7f5022200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f501c1047e0 0x7f501c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.836+0000 7f5022200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f501c108f60 0x7f501c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:46.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5022200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f501c113fc0 con 0x7f501c103e30 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5022200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f501c113e40 con 0x7f501c108f60 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5022200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f501c114140 con 0x7f501c1047e0 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5020e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c103e30 0x7f501c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5020e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c103e30 0x7f501c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56166/0 (socket says 172.21.15.103:56166) 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5020e00640 1 -- 172.21.15.103:0/3225249103 learned_addr learned my addr 172.21.15.103:0/3225249103 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f501be00640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f501c1047e0 0x7f501c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5021800640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f501c108f60 0x7f501c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5020e00640 1 -- 172.21.15.103:0/3225249103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f501c1047e0 msgr2=0x7f501c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5020e00640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f501c1047e0 0x7f501c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5020e00640 1 -- 172.21.15.103:0/3225249103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f501c108f60 msgr2=0x7f501c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:46.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5020e00640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f501c108f60 0x7f501c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:46.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.837+0000 7f5020e00640 1 -- 172.21.15.103:0/3225249103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c19f450 con 0x7f501c103e30 2024-09-16T02:07:46.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.838+0000 7f5020e00640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c103e30 0x7f501c19a4a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f501000e9f0 tx=0x7f501000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:46.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.838+0000 7f5019600640 1 -- 172.21.15.103:0/3225249103 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501000ccc0 con 0x7f501c103e30 2024-09-16T02:07:46.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.838+0000 7f5019600640 1 -- 172.21.15.103:0/3225249103 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f501000ce60 con 0x7f501c103e30 2024-09-16T02:07:46.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.838+0000 7f5022200640 1 -- 172.21.15.103:0/3225249103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501c19f740 con 0x7f501c103e30 2024-09-16T02:07:46.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.838+0000 7f5019600640 1 -- 172.21.15.103:0/3225249103 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010010690 con 0x7f501c103e30 2024-09-16T02:07:46.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.838+0000 7f5022200640 1 -- 172.21.15.103:0/3225249103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501c1a7020 con 0x7f501c103e30 2024-09-16T02:07:46.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.839+0000 7f5022200640 1 -- 172.21.15.103:0/3225249103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501c104210 con 0x7f501c103e30 2024-09-16T02:07:46.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.841+0000 7f5019600640 1 -- 172.21.15.103:0/3225249103 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f50100046c0 con 0x7f501c103e30 2024-09-16T02:07:46.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.842+0000 7f5019600640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f50040a3240 0x7f50040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:46.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.842+0000 7f5019600640 1 -- 172.21.15.103:0/3225249103 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f50100cfe10 con 0x7f501c103e30 2024-09-16T02:07:46.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.843+0000 7f501be00640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f50040a3240 0x7f50040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:46.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.843+0000 7f501be00640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f50040a3240 0x7f50040a5700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5008009230 tx=0x7f50080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:46.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:46.846+0000 7f5019600640 1 -- 172.21.15.103:0/3225249103 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50100d3030 con 0x7f501c103e30 2024-09-16T02:07:47.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.054+0000 7f5022200640 1 -- 172.21.15.103:0/3225249103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f501c10c1a0 con 0x7f501c103e30 2024-09-16T02:07:47.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.055+0000 7f5019600640 1 -- 172.21.15.103:0/3225249103 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7f501009ca50 con 0x7f501c103e30 2024-09-16T02:07:47.057 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:07:47.058 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":24,"btime":"2024-09-16T02:05:32:550231+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:05:31.661572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24506,24505]},"id":1}]} 2024-09-16T02:07:47.058 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 24 2024-09-16T02:07:47.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 -- 172.21.15.103:0/3225249103 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f50040a3240 msgr2=0x7f50040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f50040a3240 0x7f50040a5700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5008009230 tx=0x7f50080058d0 comp rx=0 tx=0).stop 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 -- 172.21.15.103:0/3225249103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c103e30 msgr2=0x7f501c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c103e30 0x7f501c19a4a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f501000e9f0 tx=0x7f501000edf0 comp rx=0 tx=0).stop 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 -- 172.21.15.103:0/3225249103 shutdown_connections 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f50040a3240 0x7f50040a5700 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f501c108f60 0x7f501c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f501c1047e0 0x7f501c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 --2- 172.21.15.103:0/3225249103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c103e30 0x7f501c19a4a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 -- 172.21.15.103:0/3225249103 >> 172.21.15.103:0/3225249103 conn(0x7f501c0ff8d0 msgr2=0x7f501c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.057+0000 7f5022200640 1 -- 172.21.15.103:0/3225249103 shutdown_connections 2024-09-16T02:07:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:47.058+0000 7f5022200640 1 -- 172.21.15.103:0/3225249103 wait complete. 2024-09-16T02:07:47.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T02:07:47.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:47 smithi103 ceph-mon[30398]: pgmap v410: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T02:07:47.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:47 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3225249103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:07:48.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:47 smithi136 ceph-mon[35323]: pgmap v410: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T02:07:48.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:47 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3225249103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:07:48.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:47 smithi172 ceph-mon[35340]: pgmap v410: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T02:07:48.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:47 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3225249103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:07:48.202 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:48.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.795+0000 7f9992800640 1 -- 172.21.15.103:0/2717166769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f998c101de0 msgr2=0x7f998c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:48.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.795+0000 7f9992800640 1 --2- 172.21.15.103:0/2717166769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f998c101de0 0x7f998c1021c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f997c009900 tx=0x7f997c02a410 comp rx=0 tx=0).stop 2024-09-16T02:07:48.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.796+0000 7f9992800640 1 -- 172.21.15.103:0/2717166769 shutdown_connections 2024-09-16T02:07:48.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.796+0000 7f9992800640 1 --2- 172.21.15.103:0/2717166769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f998c106f10 0x7f998c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.796+0000 7f9992800640 1 --2- 172.21.15.103:0/2717166769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f998c102790 0x7f998c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.796+0000 7f9992800640 1 --2- 172.21.15.103:0/2717166769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f998c101de0 0x7f998c1021c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.796+0000 7f9992800640 1 -- 172.21.15.103:0/2717166769 >> 172.21.15.103:0/2717166769 conn(0x7f998c0fd8c0 msgr2=0x7f998c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:48.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.796+0000 7f9992800640 1 -- 172.21.15.103:0/2717166769 shutdown_connections 2024-09-16T02:07:48.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.796+0000 7f9992800640 1 -- 172.21.15.103:0/2717166769 wait complete. 2024-09-16T02:07:48.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.797+0000 7f9992800640 1 Processor -- start 2024-09-16T02:07:48.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.797+0000 7f9992800640 1 -- start start 2024-09-16T02:07:48.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.797+0000 7f9992800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f998c101de0 0x7f998c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.797+0000 7f9992800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f998c102790 0x7f998c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.797+0000 7f9992800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f998c106f10 0x7f998c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.797+0000 7f9992800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f998c111fc0 con 0x7f998c106f10 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.797+0000 7f9992800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f998c111e40 con 0x7f998c101de0 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.797+0000 7f9992800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f998c112140 con 0x7f998c102790 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.797+0000 7f998be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f998c106f10 0x7f998c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.797+0000 7f998be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f998c106f10 0x7f998c19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53448/0 (socket says 172.21.15.103:53448) 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.798+0000 7f998be00640 1 -- 172.21.15.103:0/793297454 learned_addr learned my addr 172.21.15.103:0/793297454 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.798+0000 7f998aa00640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f998c102790 0x7f998c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.798+0000 7f998b400640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f998c101de0 0x7f998c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.798+0000 7f998be00640 1 -- 172.21.15.103:0/793297454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f998c102790 msgr2=0x7f998c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.798+0000 7f998be00640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f998c102790 0x7f998c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.798+0000 7f998be00640 1 -- 172.21.15.103:0/793297454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f998c101de0 msgr2=0x7f998c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.798+0000 7f998be00640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f998c101de0 0x7f998c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.798+0000 7f998be00640 1 -- 172.21.15.103:0/793297454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c19f5e0 con 0x7f998c106f10 2024-09-16T02:07:48.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.798+0000 7f998be00640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f998c106f10 0x7f998c19ef00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f998400eb10 tx=0x7f998400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:48.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.799+0000 7f9983e00640 1 -- 172.21.15.103:0/793297454 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984004b20 con 0x7f998c106f10 2024-09-16T02:07:48.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.799+0000 7f9983e00640 1 -- 172.21.15.103:0/793297454 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9984004cc0 con 0x7f998c106f10 2024-09-16T02:07:48.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.799+0000 7f9992800640 1 -- 172.21.15.103:0/793297454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c19f8d0 con 0x7f998c106f10 2024-09-16T02:07:48.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.799+0000 7f9983e00640 1 -- 172.21.15.103:0/793297454 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984010760 con 0x7f998c106f10 2024-09-16T02:07:48.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.799+0000 7f9992800640 1 -- 172.21.15.103:0/793297454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c1a71b0 con 0x7f998c106f10 2024-09-16T02:07:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.802+0000 7f9983e00640 1 -- 172.21.15.103:0/793297454 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9984005000 con 0x7f998c106f10 2024-09-16T02:07:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.802+0000 7f999541d740 1 Processor -- start 2024-09-16T02:07:48.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.802+0000 7f999541d740 1 -- start start 2024-09-16T02:07:48.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.802+0000 7f999541d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 0x56409529d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.802+0000 7f999541d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.802+0000 7f999541d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.802+0000 7f999541d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5640952033e0 con 0x56409529d590 2024-09-16T02:07:48.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.802+0000 7f999541d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5640951aad80 con 0x56409529cc70 2024-09-16T02:07:48.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.802+0000 7f999541d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5640951ff3a0 con 0x5640952a7e50 2024-09-16T02:07:48.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.802+0000 7f998b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53452/0 (socket says 172.21.15.103:53452) 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998b400640 1 -- 172.21.15.103:0/1737085271 learned_addr learned my addr 172.21.15.103:0/1737085271 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998aa00640 1 --2- 172.21.15.103:0/1737085271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 0x56409529d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998aa00640 1 --2- 172.21.15.103:0/1737085271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998b400640 1 -- 172.21.15.103:0/1737085271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 msgr2=0x5640952aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998b400640 1 --2- 172.21.15.103:0/1737085271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998b400640 1 -- 172.21.15.103:0/1737085271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 msgr2=0x56409529d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998b400640 1 --2- 172.21.15.103:0/1737085271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 0x56409529d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998b400640 1 -- 172.21.15.103:0/1737085271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640951c9ae0 con 0x56409529d590 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998aa00640 1 --2- 172.21.15.103:0/1737085271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 0x56409529d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f998b400640 1 --2- 172.21.15.103:0/1737085271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952a7910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f997c009900 tx=0x7f997c005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40768a074ae6c3be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.803+0000 7f9983e00640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99700a3240 0x7f99700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.804+0000 7f9983e00640 1 -- 172.21.15.103:0/793297454 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f99840cfe70 con 0x7f998c106f10 2024-09-16T02:07:48.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.804+0000 7f9980c00640 1 -- 172.21.15.103:0/1737085271 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c035040 con 0x56409529d590 2024-09-16T02:07:48.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.804+0000 7f9980c00640 1 -- 172.21.15.103:0/1737085271 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997c004400 con 0x56409529d590 2024-09-16T02:07:48.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.804+0000 7f9980c00640 1 -- 172.21.15.103:0/1737085271 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c0035f0 con 0x56409529d590 2024-09-16T02:07:48.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.804+0000 7f998aa00640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99700a3240 0x7f99700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.804+0000 7f999541d740 1 -- 172.21.15.103:0/1737085271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 msgr2=0x5640952a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.804+0000 7f999541d740 1 --2- 172.21.15.103:0/1737085271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952a7910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f997c009900 tx=0x7f997c005de0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.804+0000 7f998aa00640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99700a3240 0x7f99700a5700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x56409529a580 tx=0x7f99780096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.805+0000 7f999541d740 1 -- 172.21.15.103:0/1737085271 shutdown_connections 2024-09-16T02:07:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.805+0000 7f999541d740 1 --2- 172.21.15.103:0/1737085271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.805+0000 7f999541d740 1 --2- 172.21.15.103:0/1737085271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952a7910 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.805+0000 7f999541d740 1 --2- 172.21.15.103:0/1737085271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 0x56409529d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.805+0000 7f999541d740 1 -- 172.21.15.103:0/1737085271 >> 172.21.15.103:0/1737085271 conn(0x564095299200 msgr2=0x5640952995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.805+0000 7f999541d740 1 -- 172.21.15.103:0/1737085271 shutdown_connections 2024-09-16T02:07:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.805+0000 7f999541d740 1 -- 172.21.15.103:0/1737085271 wait complete. 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.805+0000 7f999541d740 1 -- start start 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.805+0000 7f999541d740 1 Processor -- start 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f999541d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529cc70 0x5640952a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f999541d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529d590 0x5640952a5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f999541d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f999541d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5640952aaa20 con 0x56409529cc70 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f999541d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5640951be0f0 con 0x56409529d590 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f999541d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5640952aaba0 con 0x5640952a7e50 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f999541d740 10 client.0 _ll_register_callbacks cb 0x5640951a5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f998b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529cc70 0x5640952a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f998b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529cc70 0x5640952a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53462/0 (socket says 172.21.15.103:53462) 2024-09-16T02:07:48.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f998b400640 1 -- 172.21.15.103:0/2354017884 learned_addr learned my addr 172.21.15.103:0/2354017884 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f998be00640 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.806+0000 7f998aa00640 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529d590 0x5640952a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.807+0000 7f998b400640 1 -- 172.21.15.103:0/2354017884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 msgr2=0x5640952a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.807+0000 7f998b400640 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.807+0000 7f998b400640 1 -- 172.21.15.103:0/2354017884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529d590 msgr2=0x5640952a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.807+0000 7f998b400640 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529d590 0x5640952a5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.807+0000 7f998b400640 1 -- 172.21.15.103:0/2354017884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564094f743c0 con 0x56409529cc70 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.807+0000 7f998b400640 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529cc70 0x5640952a4d30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f997c004920 tx=0x7f997c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47ae2f04daa1576 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.807+0000 7f9975600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.807+0000 7f9975600640 1 -- 172.21.15.103:0/2354017884 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c052060 con 0x56409529cc70 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.807+0000 7f999541d740 1 -- 172.21.15.103:0/2354017884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f997c004400 con 0x56409529cc70 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.807+0000 7f9975600640 1 -- 172.21.15.103:0/2354017884 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f997c004200 con 0x56409529cc70 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f9975600640 1 -- 172.21.15.103:0/2354017884 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f997c035070 con 0x56409529cc70 2024-09-16T02:07:48.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f999541d740 10 client.15492 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f999541d740 1 -- 172.21.15.103:0/2354017884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5640952b6a60 con 0x56409529cc70 2024-09-16T02:07:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f9975600640 1 -- 172.21.15.103:0/2354017884 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997c004400 con 0x56409529cc70 2024-09-16T02:07:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f9975600640 1 -- 172.21.15.103:0/2354017884 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f997c031b00 con 0x56409529cc70 2024-09-16T02:07:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f999541d740 10 client.15492 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f999541d740 10 client.15492 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T02:07:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f999541d740 20 client.15492 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f999541d740 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x5640952b7db0 0x5640952ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f999541d740 4 client.15492 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.808+0000 7f999541d740 1 -- 172.21.15.103:0/2354017884 --> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564095210950 con 0x5640952b7db0 2024-09-16T02:07:48.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.809+0000 7f998aa00640 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x5640952b7db0 0x5640952ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.809+0000 7f998aa00640 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x5640952b7db0 0x5640952ba1a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97a700964b95961b server_cookie=29a85b44315edbaf in_seq=0 out_seq=0 2024-09-16T02:07:48.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.809+0000 7f9975600640 10 client.15492 ms_handle_connect on v2:172.21.15.172:6838/3474994563 2024-09-16T02:07:48.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f9975600640 1 -- 172.21.15.103:0/2354017884 <== mds.2 v2:172.21.15.172:6838/3474994563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564095210950 con 0x5640952b7db0 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f9975600640 10 client.15492 handle_command_reply: tid=0 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 client.15492 shutdown 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 -- 172.21.15.103:0/2354017884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529cc70 msgr2=0x5640952a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529cc70 0x5640952a4d30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f997c004920 tx=0x7f997c004af0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 -- 172.21.15.103:0/2354017884 shutdown_connections 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x5640952b7db0 0x5640952ba1a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529d590 0x5640952a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 --2- 172.21.15.103:0/2354017884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529cc70 0x5640952a4d30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 -- 172.21.15.103:0/2354017884 >> 172.21.15.103:0/2354017884 conn(0x564095299200 msgr2=0x564095298380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f9975600640 0 client.15492 ms_handle_reset on v2:172.21.15.172:6838/3474994563 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 -- 172.21.15.103:0/2354017884 shutdown_connections 2024-09-16T02:07:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 1 -- 172.21.15.103:0/2354017884 wait complete. 2024-09-16T02:07:48.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.811+0000 7f999541d740 20 client.15492 trim_cache size 0 max 16384 2024-09-16T02:07:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.825+0000 7f999541d740 1 Processor -- start 2024-09-16T02:07:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.825+0000 7f999541d740 1 -- start start 2024-09-16T02:07:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.825+0000 7f999541d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 0x5640952a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.825+0000 7f999541d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952bf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.825+0000 7f999541d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952bfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.825+0000 7f999541d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1b2280 con 0x56409529d590 2024-09-16T02:07:48.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.825+0000 7f999541d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1afb50 con 0x56409529cc70 2024-09-16T02:07:48.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.825+0000 7f999541d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1ad9a0 con 0x5640952a7e50 2024-09-16T02:07:48.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952bf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 0x5640952a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952bf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53468/0 (socket says 172.21.15.103:53468) 2024-09-16T02:07:48.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998aa00640 1 -- 172.21.15.103:0/914558295 learned_addr learned my addr 172.21.15.103:0/914558295 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998be00640 1 --2- 172.21.15.103:0/914558295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952bfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998b400640 1 -- 172.21.15.103:0/914558295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 msgr2=0x5640952bfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998b400640 1 --2- 172.21.15.103:0/914558295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952bfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998b400640 1 -- 172.21.15.103:0/914558295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 msgr2=0x5640952bf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998b400640 1 --2- 172.21.15.103:0/914558295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952bf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998b400640 1 -- 172.21.15.103:0/914558295 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564095294770 con 0x56409529cc70 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998be00640 1 --2- 172.21.15.103:0/914558295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952bfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998aa00640 1 --2- 172.21.15.103:0/914558295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952bf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f998b400640 1 --2- 172.21.15.103:0/914558295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 0x5640952a6ca0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f997c0048f0 tx=0x7f997c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8da15e5885cbe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f9977e00640 1 -- 172.21.15.103:0/914558295 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c052030 con 0x56409529cc70 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f9977e00640 1 -- 172.21.15.103:0/914558295 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997c003ce0 con 0x56409529cc70 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.826+0000 7f9977e00640 1 -- 172.21.15.103:0/914558295 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c031810 con 0x56409529cc70 2024-09-16T02:07:48.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.827+0000 7f999541d740 1 -- 172.21.15.103:0/914558295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 msgr2=0x5640952a6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:48.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.827+0000 7f999541d740 1 --2- 172.21.15.103:0/914558295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 0x5640952a6ca0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f997c0048f0 tx=0x7f997c000bc0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.827+0000 7f999541d740 1 -- 172.21.15.103:0/914558295 shutdown_connections 2024-09-16T02:07:48.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.827+0000 7f999541d740 1 --2- 172.21.15.103:0/914558295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640952a7e50 0x5640952bfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.827+0000 7f999541d740 1 --2- 172.21.15.103:0/914558295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952bf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.827+0000 7f999541d740 1 --2- 172.21.15.103:0/914558295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56409529cc70 0x5640952a6ca0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.827+0000 7f999541d740 1 -- 172.21.15.103:0/914558295 >> 172.21.15.103:0/914558295 conn(0x564095299200 msgr2=0x5640952a6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:48.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.827+0000 7f999541d740 1 -- 172.21.15.103:0/914558295 shutdown_connections 2024-09-16T02:07:48.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.827+0000 7f999541d740 1 -- 172.21.15.103:0/914558295 wait complete. 2024-09-16T02:07:48.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.828+0000 7f999541d740 1 -- start start 2024-09-16T02:07:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.828+0000 7f999541d740 1 Processor -- start 2024-09-16T02:07:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.828+0000 7f999541d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56409529cc70 0x5640952b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.828+0000 7f999541d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952a1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.828+0000 7f999541d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5640952a7e50 0x5640952a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.828+0000 7f999541d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1ae010 con 0x56409529d590 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.828+0000 7f999541d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1af480 con 0x5640952a7e50 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.828+0000 7f999541d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1ae6e0 con 0x56409529cc70 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.828+0000 7f999541d740 10 client.0 _ll_register_callbacks cb 0x7f997c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5640952a7e50 0x5640952a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5640952a7e50 0x5640952a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:47480/0 (socket says 172.21.15.103:47480) 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998be00640 1 -- 172.21.15.103:0/3920307449 learned_addr learned my addr 172.21.15.103:0/3920307449 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998b400640 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56409529cc70 0x5640952b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998be00640 1 -- 172.21.15.103:0/3920307449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56409529cc70 msgr2=0x5640952b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998be00640 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56409529cc70 0x5640952b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998be00640 1 -- 172.21.15.103:0/3920307449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 msgr2=0x5640952a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998be00640 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998be00640 1 -- 172.21.15.103:0/3920307449 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56409529fae0 con 0x5640952a7e50 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998b400640 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56409529cc70 0x5640952b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f998be00640 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5640952a7e50 0x5640952a1e80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f998409c380 tx=0x7f998409bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e1ecf4d992548b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f9975600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f9975600640 1 -- 172.21.15.103:0/3920307449 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840b4020 con 0x5640952a7e50 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f9975600640 1 -- 172.21.15.103:0/3920307449 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f99840a4bd0 con 0x5640952a7e50 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.829+0000 7f999541d740 1 -- 172.21.15.103:0/3920307449 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f997c003ce0 con 0x5640952a7e50 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f9975600640 1 -- 172.21.15.103:0/3920307449 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f998407f050 con 0x5640952a7e50 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f999541d740 10 client.25475 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f999541d740 1 -- 172.21.15.103:0/3920307449 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564095322400 con 0x5640952a7e50 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f9975600640 1 -- 172.21.15.103:0/3920307449 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997c003ce0 con 0x5640952a7e50 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f9975600640 1 -- 172.21.15.103:0/3920307449 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f99840052f0 con 0x5640952a7e50 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f999541d740 10 client.25475 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f999541d740 10 client.25475 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f999541d740 20 client.25475 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f999541d740 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564095322bd0 0x564095324ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f999541d740 4 client.25475 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f999541d740 1 -- 172.21.15.103:0/3920307449 --> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56409520ae60 con 0x564095322bd0 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.830+0000 7f998b400640 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564095322bd0 0x564095324ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f998b400640 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564095322bd0 0x564095324ff0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e43338f6acf856f7 server_cookie=b0fbfb8a9a915844 in_seq=0 out_seq=0 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f9975600640 10 client.25475 ms_handle_connect on v2:172.21.15.172:6838/3474994563 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f9975600640 1 -- 172.21.15.103:0/3920307449 <== mds.2 v2:172.21.15.172:6838/3474994563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56409520ae60 con 0x564095322bd0 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f9975600640 10 client.25475 handle_command_reply: tid=0 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f999541d740 1 client.25475 shutdown 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f999541d740 1 -- 172.21.15.103:0/3920307449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5640952a7e50 msgr2=0x5640952a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f999541d740 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5640952a7e50 0x5640952a1e80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f998409c380 tx=0x7f998409bfb0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f999541d740 1 -- 172.21.15.103:0/3920307449 shutdown_connections 2024-09-16T02:07:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f999541d740 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.172:6838/3474994563,v1:172.21.15.172:6839/3474994563] conn(0x564095322bd0 0x564095324ff0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f999541d740 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5640952a7e50 0x5640952a1e80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f999541d740 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56409529d590 0x5640952a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f999541d740 1 --2- 172.21.15.103:0/3920307449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56409529cc70 0x5640952b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f999541d740 1 -- 172.21.15.103:0/3920307449 >> 172.21.15.103:0/3920307449 conn(0x564095299200 msgr2=0x5640952a6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.831+0000 7f9975600640 0 client.25475 ms_handle_reset on v2:172.21.15.172:6838/3474994563 2024-09-16T02:07:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f999541d740 1 -- 172.21.15.103:0/3920307449 shutdown_connections 2024-09-16T02:07:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f999541d740 1 -- 172.21.15.103:0/3920307449 wait complete. 2024-09-16T02:07:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f999541d740 20 client.25475 trim_cache size 0 max 16384 2024-09-16T02:07:48.840 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-16T02:07:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f9992800640 1 -- 172.21.15.103:0/793297454 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99700a3240 msgr2=0x7f99700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f9992800640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99700a3240 0x7f99700a5700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x56409529a580 tx=0x7f99780096c0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f9992800640 1 -- 172.21.15.103:0/793297454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f998c106f10 msgr2=0x7f998c19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f9992800640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f998c106f10 0x7f998c19ef00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f998400eb10 tx=0x7f998400c2d0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f9992800640 1 -- 172.21.15.103:0/793297454 shutdown_connections 2024-09-16T02:07:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f9992800640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f99700a3240 0x7f99700a5700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f9992800640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f998c106f10 0x7f998c19ef00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f9992800640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f998c102790 0x7f998c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f9992800640 1 --2- 172.21.15.103:0/793297454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f998c101de0 0x7f998c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.832+0000 7f9992800640 1 -- 172.21.15.103:0/793297454 >> 172.21.15.103:0/793297454 conn(0x7f998c0fd8c0 msgr2=0x7f998c0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.833+0000 7f9992800640 1 -- 172.21.15.103:0/793297454 shutdown_connections 2024-09-16T02:07:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:48.833+0000 7f9992800640 1 -- 172.21.15.103:0/793297454 wait complete. 2024-09-16T02:07:49.645 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T02:07:50.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:49 smithi103 ceph-mon[30398]: pgmap v411: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T02:07:50.020 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:50.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:49 smithi136 ceph-mon[35323]: pgmap v411: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T02:07:50.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:49 smithi172 ceph-mon[35340]: pgmap v411: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-16T02:07:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.564+0000 7fe561800640 1 -- 172.21.15.103:0/109916948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103ff0 msgr2=0x7fe55c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.564+0000 7fe561800640 1 --2- 172.21.15.103:0/109916948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103ff0 0x7fe55c1043d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009900 tx=0x7fe54c02a3f0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.564+0000 7fe561800640 1 -- 172.21.15.103:0/109916948 shutdown_connections 2024-09-16T02:07:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.564+0000 7fe561800640 1 --2- 172.21.15.103:0/109916948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c109120 0x7fe55c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.564+0000 7fe561800640 1 --2- 172.21.15.103:0/109916948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1049a0 0x7fe55c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.564+0000 7fe561800640 1 --2- 172.21.15.103:0/109916948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103ff0 0x7fe55c1043d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.564+0000 7fe561800640 1 -- 172.21.15.103:0/109916948 >> 172.21.15.103:0/109916948 conn(0x7fe55c0ffa90 msgr2=0x7fe55c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.564+0000 7fe561800640 1 -- 172.21.15.103:0/109916948 shutdown_connections 2024-09-16T02:07:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.564+0000 7fe561800640 1 -- 172.21.15.103:0/109916948 wait complete. 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.565+0000 7fe561800640 1 Processor -- start 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.565+0000 7fe561800640 1 -- start start 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe561800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe561800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1049a0 0x7fe55c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe561800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c109120 0x7fe55c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe561800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c114140 con 0x7fe55c109120 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe561800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c113fc0 con 0x7fe55c1049a0 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe561800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1142c0 con 0x7fe55c103ff0 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe55aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe55aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:33900/0 (socket says 172.21.15.103:33900) 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe55aa00640 1 -- 172.21.15.103:0/2195904298 learned_addr learned my addr 172.21.15.103:0/2195904298 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe55b400640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c109120 0x7fe55c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe55a000640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1049a0 0x7fe55c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe55aa00640 1 -- 172.21.15.103:0/2195904298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1049a0 msgr2=0x7fe55c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe55aa00640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1049a0 0x7fe55c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe55aa00640 1 -- 172.21.15.103:0/2195904298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c109120 msgr2=0x7fe55c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe55aa00640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c109120 0x7fe55c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.566+0000 7fe55aa00640 1 -- 172.21.15.103:0/2195904298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c19f5b0 con 0x7fe55c103ff0 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.567+0000 7fe55aa00640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe54c006010 tx=0x7fe54c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.567+0000 7fe553400640 1 -- 172.21.15.103:0/2195904298 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c03f030 con 0x7fe55c103ff0 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.567+0000 7fe553400640 1 -- 172.21.15.103:0/2195904298 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c004440 con 0x7fe55c103ff0 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.567+0000 7fe553400640 1 -- 172.21.15.103:0/2195904298 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c035070 con 0x7fe55c103ff0 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.567+0000 7fe561800640 1 -- 172.21.15.103:0/2195904298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c19f840 con 0x7fe55c103ff0 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.568+0000 7fe561800640 1 -- 172.21.15.103:0/2195904298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c19fcd0 con 0x7fe55c103ff0 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.571+0000 7fe5641dc740 1 Processor -- start 2024-09-16T02:07:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.572+0000 7fe5641dc740 1 -- start start 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.572+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493ec70 0x560d6493f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.572+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 0x560d64949910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.572+0000 7fe553400640 1 -- 172.21.15.103:0/2195904298 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe54c042030 con 0x7fe55c103ff0 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.572+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d64949e50 0x560d6494c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.572+0000 7fe5641dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560d648a53e0 con 0x560d6493f590 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.572+0000 7fe5641dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560d6484cd80 con 0x560d64949e50 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.572+0000 7fe5641dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560d648a13a0 con 0x560d6493ec70 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493ec70 0x560d6493f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493ec70 0x560d6493f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:33916/0 (socket says 172.21.15.103:33916) 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55b400640 1 -- 172.21.15.103:0/1973866117 learned_addr learned my addr 172.21.15.103:0/1973866117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d64949e50 0x560d6494c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55aa00640 1 --2- 172.21.15.103:0/1973866117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 0x560d64949910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55b400640 1 -- 172.21.15.103:0/1973866117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d64949e50 msgr2=0x560d6494c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55b400640 1 --2- 172.21.15.103:0/1973866117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d64949e50 0x560d6494c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55b400640 1 -- 172.21.15.103:0/1973866117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 msgr2=0x560d64949910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55b400640 1 --2- 172.21.15.103:0/1973866117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 0x560d64949910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55b400640 1 -- 172.21.15.103:0/1973866117 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6486bae0 con 0x560d6493ec70 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.573+0000 7fe55b400640 1 --2- 172.21.15.103:0/1973866117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493ec70 0x560d6493f050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe554009f40 tx=0x7fe554010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58aaadf1785a3ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.574+0000 7fe547e00640 1 -- 172.21.15.103:0/1973866117 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554003b70 con 0x560d6493ec70 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.574+0000 7fe547e00640 1 -- 172.21.15.103:0/1973866117 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe554003d10 con 0x560d6493ec70 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.574+0000 7fe547e00640 1 -- 172.21.15.103:0/1973866117 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554011af0 con 0x560d6493ec70 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.574+0000 7fe553400640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe5400a32c0 0x7fe5400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe553400640 1 -- 172.21.15.103:0/2195904298 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fe54c0efe10 con 0x7fe55c103ff0 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe55aa00640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe5400a32c0 0x7fe5400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe5641dc740 1 -- 172.21.15.103:0/1973866117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493ec70 msgr2=0x560d6493f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:50.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe5641dc740 1 --2- 172.21.15.103:0/1973866117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493ec70 0x560d6493f050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe554009f40 tx=0x7fe554010860 comp rx=0 tx=0).stop 2024-09-16T02:07:50.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe5641dc740 1 -- 172.21.15.103:0/1973866117 shutdown_connections 2024-09-16T02:07:50.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe5641dc740 1 --2- 172.21.15.103:0/1973866117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d64949e50 0x560d6494c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe5641dc740 1 --2- 172.21.15.103:0/1973866117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 0x560d64949910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe5641dc740 1 --2- 172.21.15.103:0/1973866117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493ec70 0x560d6493f050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe5641dc740 1 -- 172.21.15.103:0/1973866117 >> 172.21.15.103:0/1973866117 conn(0x560d6493b200 msgr2=0x560d6493b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe5641dc740 1 -- 172.21.15.103:0/1973866117 shutdown_connections 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe55aa00640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe5400a32c0 0x7fe5400a5780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe54c005ab0 tx=0x7fe54c0daf90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.575+0000 7fe5641dc740 1 -- 172.21.15.103:0/1973866117 wait complete. 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.576+0000 7fe5641dc740 1 -- start start 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.576+0000 7fe5641dc740 1 Processor -- start 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 0x560d64946da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493f590 0x560d649472e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d64949e50 0x560d64947820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe5641dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560d6494ca20 con 0x560d64949e50 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe5641dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560d648600f0 con 0x560d6493ec70 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe5641dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560d6494cba0 con 0x560d6493f590 2024-09-16T02:07:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe5641dc740 10 client.0 _ll_register_callbacks cb 0x560d64847bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe55b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 0x560d64946da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe55a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493f590 0x560d649472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe55a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493f590 0x560d649472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:33928/0 (socket says 172.21.15.103:33928) 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe55a000640 1 -- 172.21.15.103:0/395280939 learned_addr learned my addr 172.21.15.103:0/395280939 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe55b400640 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d64949e50 0x560d64947820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe55a000640 1 -- 172.21.15.103:0/395280939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 msgr2=0x560d64946da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe55a000640 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 0x560d64946da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe55a000640 1 -- 172.21.15.103:0/395280939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d64949e50 msgr2=0x560d64947820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe55a000640 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d64949e50 0x560d64947820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.577+0000 7fe55a000640 1 -- 172.21.15.103:0/395280939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d646163c0 con 0x560d6493f590 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.578+0000 7fe55b400640 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d64949e50 0x560d64947820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.578+0000 7fe55a000640 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493f590 0x560d649472e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe548012730 tx=0x7fe548012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76aceee1cc3c754e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.578+0000 7fe544c00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.578+0000 7fe544c00640 1 -- 172.21.15.103:0/395280939 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548011bb0 con 0x560d6493f590 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.578+0000 7fe544c00640 1 -- 172.21.15.103:0/395280939 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fe548011d50 con 0x560d6493f590 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.578+0000 7fe5641dc740 1 -- 172.21.15.103:0/395280939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe554003d10 con 0x560d6493f590 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.578+0000 7fe544c00640 1 -- 172.21.15.103:0/395280939 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe548004030 con 0x560d6493f590 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.579+0000 7fe5641dc740 10 client.25459 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.579+0000 7fe5641dc740 1 -- 172.21.15.103:0/395280939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560d64958b70 con 0x560d6493f590 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.579+0000 7fe544c00640 1 -- 172.21.15.103:0/395280939 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe554003d10 con 0x560d6493f590 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.579+0000 7fe544c00640 1 -- 172.21.15.103:0/395280939 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe54801c460 con 0x560d6493f590 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.579+0000 7fe5641dc740 10 client.25459 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.579+0000 7fe5641dc740 10 client.25459 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.580+0000 7fe5641dc740 20 client.25459 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.580+0000 7fe5641dc740 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x560d6495a080 0x560d6495c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.580+0000 7fe5641dc740 4 client.25459 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.580+0000 7fe5641dc740 1 -- 172.21.15.103:0/395280939 --> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d648b2950 con 0x560d6495a080 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.580+0000 7fe55b400640 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x560d6495a080 0x560d6495c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.580+0000 7fe55b400640 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x560d6495a080 0x560d6495c4a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e69856cf9c790526 server_cookie=3244b22eaf0a03cd in_seq=0 out_seq=0 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.580+0000 7fe544c00640 10 client.25459 ms_handle_connect on v2:172.21.15.136:6836/236571207 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.581+0000 7fe544c00640 1 -- 172.21.15.103:0/395280939 <== mds.1 v2:172.21.15.136:6836/236571207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d648b2950 con 0x560d6495a080 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.581+0000 7fe544c00640 10 client.25459 handle_command_reply: tid=0 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.581+0000 7fe5641dc740 1 client.25459 shutdown 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.581+0000 7fe5641dc740 1 -- 172.21.15.103:0/395280939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493f590 msgr2=0x560d649472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.581+0000 7fe5641dc740 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493f590 0x560d649472e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe548012730 tx=0x7fe548012bf0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.582+0000 7fe5641dc740 1 -- 172.21.15.103:0/395280939 shutdown_connections 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.582+0000 7fe5641dc740 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d64949e50 0x560d64947820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.582+0000 7fe5641dc740 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d6493f590 0x560d649472e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.582+0000 7fe5641dc740 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x560d6495a080 0x560d6495c4a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.582+0000 7fe5641dc740 1 --2- 172.21.15.103:0/395280939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 0x560d64946da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.582+0000 7fe5641dc740 1 -- 172.21.15.103:0/395280939 >> 172.21.15.103:0/395280939 conn(0x560d6493b200 msgr2=0x560d64940f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:50.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.582+0000 7fe544c00640 0 client.25459 ms_handle_reset on v2:172.21.15.136:6836/236571207 2024-09-16T02:07:50.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.582+0000 7fe5641dc740 1 -- 172.21.15.103:0/395280939 shutdown_connections 2024-09-16T02:07:50.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.582+0000 7fe5641dc740 1 -- 172.21.15.103:0/395280939 wait complete. 2024-09-16T02:07:50.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.582+0000 7fe5641dc740 20 client.25459 trim_cache size 0 max 16384 2024-09-16T02:07:50.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.595+0000 7fe5641dc740 1 Processor -- start 2024-09-16T02:07:50.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe5641dc740 1 -- start start 2024-09-16T02:07:50.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 0x560d64961d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 0x560d649622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 0x560d64948600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe5641dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1b2280 con 0x560d6493f590 2024-09-16T02:07:50.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe5641dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1afb50 con 0x560d6493ec70 2024-09-16T02:07:50.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe5641dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ad9a0 con 0x560d64949e50 2024-09-16T02:07:50.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe55a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 0x560d649622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe55b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 0x560d64961d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe55b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 0x560d64961d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:47518/0 (socket says 172.21.15.103:47518) 2024-09-16T02:07:50.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe55a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 0x560d649622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53508/0 (socket says 172.21.15.103:53508) 2024-09-16T02:07:50.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe55b400640 1 -- 172.21.15.103:0/1009751922 learned_addr learned my addr 172.21.15.103:0/1009751922 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:50.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.596+0000 7fe55b400640 1 --2- 172.21.15.103:0/1009751922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 0x560d64948600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.597+0000 7fe55b400640 1 -- 172.21.15.103:0/1009751922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 msgr2=0x560d64948600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:50.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.597+0000 7fe55b400640 1 --2- 172.21.15.103:0/1009751922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 0x560d64948600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.597+0000 7fe55b400640 1 -- 172.21.15.103:0/1009751922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 msgr2=0x560d649622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:50.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.597+0000 7fe55b400640 1 --2- 172.21.15.103:0/1009751922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 0x560d649622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.597+0000 7fe55b400640 1 -- 172.21.15.103:0/1009751922 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6493f3c0 con 0x560d6493ec70 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.597+0000 7fe55b400640 1 --2- 172.21.15.103:0/1009751922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 0x560d64961d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe554002230 tx=0x7fe554005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba4120cf2ac49f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.597+0000 7fe547400640 1 -- 172.21.15.103:0/1009751922 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554024040 con 0x560d6493ec70 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.597+0000 7fe547400640 1 -- 172.21.15.103:0/1009751922 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe55402f070 con 0x560d6493ec70 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.597+0000 7fe547400640 1 -- 172.21.15.103:0/1009751922 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55402c690 con 0x560d6493ec70 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.598+0000 7fe5641dc740 1 -- 172.21.15.103:0/1009751922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 msgr2=0x560d64961d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.598+0000 7fe5641dc740 1 --2- 172.21.15.103:0/1009751922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 0x560d64961d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe554002230 tx=0x7fe554005fd0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.598+0000 7fe5641dc740 1 -- 172.21.15.103:0/1009751922 shutdown_connections 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.598+0000 7fe5641dc740 1 --2- 172.21.15.103:0/1009751922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 0x560d64948600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.598+0000 7fe5641dc740 1 --2- 172.21.15.103:0/1009751922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493f590 0x560d649622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.598+0000 7fe5641dc740 1 --2- 172.21.15.103:0/1009751922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493ec70 0x560d64961d70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.598+0000 7fe5641dc740 1 -- 172.21.15.103:0/1009751922 >> 172.21.15.103:0/1009751922 conn(0x560d6493b200 msgr2=0x560d64945e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.598+0000 7fe5641dc740 1 -- 172.21.15.103:0/1009751922 shutdown_connections 2024-09-16T02:07:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.598+0000 7fe5641dc740 1 -- 172.21.15.103:0/1009751922 wait complete. 2024-09-16T02:07:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.599+0000 7fe5641dc740 1 -- start start 2024-09-16T02:07:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.599+0000 7fe5641dc740 1 Processor -- start 2024-09-16T02:07:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.599+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493ec70 0x560d649522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.599+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493f590 0x560d6494d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.599+0000 7fe5641dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 0x560d6494db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.599+0000 7fe5641dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ae010 con 0x560d6493ec70 2024-09-16T02:07:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.599+0000 7fe5641dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1af480 con 0x560d6493f590 2024-09-16T02:07:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.599+0000 7fe5641dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ae6e0 con 0x560d64949e50 2024-09-16T02:07:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe5641dc740 10 client.0 _ll_register_callbacks cb 0x7fe5480097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe55a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493f590 0x560d6494d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe55a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493f590 0x560d6494d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:47524/0 (socket says 172.21.15.103:47524) 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe55a000640 1 -- 172.21.15.103:0/3873654280 learned_addr learned my addr 172.21.15.103:0/3873654280 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe55b400640 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 0x560d6494db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe55a000640 1 -- 172.21.15.103:0/3873654280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 msgr2=0x560d6494db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe55a000640 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 0x560d6494db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe55a000640 1 -- 172.21.15.103:0/3873654280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493ec70 msgr2=0x560d649522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe55a000640 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493ec70 0x560d649522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe55a000640 1 -- 172.21.15.103:0/3873654280 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6494e0d0 con 0x560d6493f590 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.600+0000 7fe55b400640 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 0x560d6494db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.601+0000 7fe55a000640 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493f590 0x560d6494d650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x560d64946630 tx=0x7fe548009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5a1a1e996960115 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.601+0000 7fe544c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-16T02:07:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.601+0000 7fe544c00640 1 -- 172.21.15.103:0/3873654280 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54802b070 con 0x560d6493f590 2024-09-16T02:07:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.601+0000 7fe544c00640 1 -- 172.21.15.103:0/3873654280 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fe548005040 con 0x560d6493f590 2024-09-16T02:07:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.601+0000 7fe5641dc740 1 -- 172.21.15.103:0/3873654280 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe55402f070 con 0x560d6493f590 2024-09-16T02:07:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.601+0000 7fe544c00640 1 -- 172.21.15.103:0/3873654280 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe548011bb0 con 0x560d6493f590 2024-09-16T02:07:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.601+0000 7fe5641dc740 10 client.25493 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.601+0000 7fe5641dc740 1 -- 172.21.15.103:0/3873654280 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560d649c4ac0 con 0x560d6493f590 2024-09-16T02:07:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.601+0000 7fe544c00640 1 -- 172.21.15.103:0/3873654280 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe55402f070 con 0x560d6493f590 2024-09-16T02:07:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.602+0000 7fe544c00640 1 -- 172.21.15.103:0/3873654280 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe548003b50 con 0x560d6493f590 2024-09-16T02:07:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.602+0000 7fe5641dc740 10 client.25493 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.602+0000 7fe5641dc740 10 client.25493 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.602+0000 7fe5641dc740 20 client.25493 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.602+0000 7fe5641dc740 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x560d649c5200 0x560d649c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.602+0000 7fe5641dc740 4 client.25493 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.602+0000 7fe5641dc740 1 -- 172.21.15.103:0/3873654280 --> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d648ace60 con 0x560d649c5200 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.602+0000 7fe55b400640 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x560d649c5200 0x560d649c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.603+0000 7fe55b400640 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x560d649c5200 0x560d649c7620 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f99c4a4021304a52 server_cookie=59e31bfcb2f9f0ef in_seq=0 out_seq=0 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.603+0000 7fe544c00640 10 client.25493 ms_handle_connect on v2:172.21.15.136:6836/236571207 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.603+0000 7fe544c00640 1 -- 172.21.15.103:0/3873654280 <== mds.1 v2:172.21.15.136:6836/236571207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d648ace60 con 0x560d649c5200 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.603+0000 7fe544c00640 10 client.25493 handle_command_reply: tid=0 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.603+0000 7fe5641dc740 1 client.25493 shutdown 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 1 -- 172.21.15.103:0/3873654280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493f590 msgr2=0x560d6494d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493f590 0x560d6494d650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x560d64946630 tx=0x7fe548009fb0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 1 -- 172.21.15.103:0/3873654280 shutdown_connections 2024-09-16T02:07:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d64949e50 0x560d6494db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.136:6836/236571207,v1:172.21.15.136:6837/236571207] conn(0x560d649c5200 0x560d649c7620 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d6493f590 0x560d6494d650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 1 --2- 172.21.15.103:0/3873654280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d6493ec70 0x560d649522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 1 -- 172.21.15.103:0/3873654280 >> 172.21.15.103:0/3873654280 conn(0x560d6493b200 msgr2=0x560d64948600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe544c00640 0 client.25493 ms_handle_reset on v2:172.21.15.136:6836/236571207 2024-09-16T02:07:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 1 -- 172.21.15.103:0/3873654280 shutdown_connections 2024-09-16T02:07:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 1 -- 172.21.15.103:0/3873654280 wait complete. 2024-09-16T02:07:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.604+0000 7fe5641dc740 20 client.25493 trim_cache size 0 max 16384 2024-09-16T02:07:50.609 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-16T02:07:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 -- 172.21.15.103:0/2195904298 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe5400a32c0 msgr2=0x7fe5400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe5400a32c0 0x7fe5400a5780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe54c005ab0 tx=0x7fe54c0daf90 comp rx=0 tx=0).stop 2024-09-16T02:07:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 -- 172.21.15.103:0/2195904298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103ff0 msgr2=0x7fe55c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe54c006010 tx=0x7fe54c004040 comp rx=0 tx=0).stop 2024-09-16T02:07:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 -- 172.21.15.103:0/2195904298 shutdown_connections 2024-09-16T02:07:50.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe5400a32c0 0x7fe5400a5780 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c109120 0x7fe55c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1049a0 0x7fe55c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 --2- 172.21.15.103:0/2195904298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a600 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:50.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 -- 172.21.15.103:0/2195904298 >> 172.21.15.103:0/2195904298 conn(0x7fe55c0ffa90 msgr2=0x7fe55c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:50.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 -- 172.21.15.103:0/2195904298 shutdown_connections 2024-09-16T02:07:50.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:50.605+0000 7fe561800640 1 -- 172.21.15.103:0/2195904298 wait complete. 2024-09-16T02:07:50.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:50 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:07:51.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:50 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:07:51.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:50 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:07:51.382 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T02:07:51.757 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:51.896 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:51 smithi103 ceph-mon[30398]: pgmap v412: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T02:07:51.896 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:07:51.896 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:07:51.897 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:07:51.897 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:07:51.897 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:51 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:07:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:51 smithi136 ceph-mon[35323]: pgmap v412: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T02:07:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:07:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:07:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:07:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:07:52.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:51 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:07:52.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:51 smithi172 ceph-mon[35340]: pgmap v412: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T02:07:52.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:07:52.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:07:52.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:07:52.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:07:52.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:51 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' 2024-09-16T02:07:52.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.340+0000 7fecaa400640 1 -- 172.21.15.103:0/3138767670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca410e260 msgr2=0x7feca4110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:52.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.340+0000 7fecaa400640 1 --2- 172.21.15.103:0/3138767670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca410e260 0x7feca4110650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009ca0 tx=0x7fec9c02a950 comp rx=0 tx=0).stop 2024-09-16T02:07:52.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.340+0000 7fecaa400640 1 -- 172.21.15.103:0/3138767670 shutdown_connections 2024-09-16T02:07:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.340+0000 7fecaa400640 1 --2- 172.21.15.103:0/3138767670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca410e260 0x7feca4110650 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.340+0000 7fecaa400640 1 --2- 172.21.15.103:0/3138767670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca4074ee0 0x7feca410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.340+0000 7fecaa400640 1 --2- 172.21.15.103:0/3138767670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca40745c0 0x7feca40749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.340+0000 7fecaa400640 1 -- 172.21.15.103:0/3138767670 >> 172.21.15.103:0/3138767670 conn(0x7feca40fd6e0 msgr2=0x7feca40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.340+0000 7fecaa400640 1 -- 172.21.15.103:0/3138767670 shutdown_connections 2024-09-16T02:07:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.340+0000 7fecaa400640 1 -- 172.21.15.103:0/3138767670 wait complete. 2024-09-16T02:07:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.341+0000 7fecaa400640 1 Processor -- start 2024-09-16T02:07:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.341+0000 7fecaa400640 1 -- start start 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.341+0000 7fecaa400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca40745c0 0x7feca419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7fecaa400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca4074ee0 0x7feca419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7fecaa400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca410e260 0x7feca419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7fecaa400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feca4113fe0 con 0x7feca40745c0 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7fecaa400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feca4113e60 con 0x7feca4074ee0 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7fecaa400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feca4114160 con 0x7feca410e260 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7feca9000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca40745c0 0x7feca419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7feca9000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca40745c0 0x7feca419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53546/0 (socket says 172.21.15.103:53546) 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7feca9000640 1 -- 172.21.15.103:0/588507556 learned_addr learned my addr 172.21.15.103:0/588507556 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7fec9be00640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca4074ee0 0x7feca419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7feca9a00640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca410e260 0x7feca419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7feca9000640 1 -- 172.21.15.103:0/588507556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca410e260 msgr2=0x7feca419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7feca9000640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca410e260 0x7feca419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7feca9000640 1 -- 172.21.15.103:0/588507556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca4074ee0 msgr2=0x7feca419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7feca9000640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca4074ee0 0x7feca419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.342+0000 7feca9000640 1 -- 172.21.15.103:0/588507556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca419f3f0 con 0x7feca40745c0 2024-09-16T02:07:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.343+0000 7feca9000640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca40745c0 0x7feca419a440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fec9400ea60 tx=0x7fec9400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.343+0000 7feca2000640 1 -- 172.21.15.103:0/588507556 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9400cde0 con 0x7feca40745c0 2024-09-16T02:07:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.343+0000 7feca2000640 1 -- 172.21.15.103:0/588507556 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec940035f0 con 0x7feca40745c0 2024-09-16T02:07:52.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.343+0000 7fecaa400640 1 -- 172.21.15.103:0/588507556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca419f6e0 con 0x7feca40745c0 2024-09-16T02:07:52.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.343+0000 7feca2000640 1 -- 172.21.15.103:0/588507556 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94010670 con 0x7feca40745c0 2024-09-16T02:07:52.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.343+0000 7fecaa400640 1 -- 172.21.15.103:0/588507556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca41a6fc0 con 0x7feca40745c0 2024-09-16T02:07:52.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.347+0000 7fecae4d7740 1 Processor -- start 2024-09-16T02:07:52.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.347+0000 7fecae4d7740 1 -- start start 2024-09-16T02:07:52.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7feca2000640 1 -- 172.21.15.103:0/588507556 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fec940037a0 con 0x7feca40745c0 2024-09-16T02:07:52.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f30e40 0x55a6a7f31220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f3bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f3c020 0x55a6a7f3e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7fecae4d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a6a7e973e0 con 0x55a6a7f31760 2024-09-16T02:07:52.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7fecae4d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6a7e3ed80 con 0x55a6a7f3c020 2024-09-16T02:07:52.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7fecae4d7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a6a7e933a0 con 0x55a6a7f30e40 2024-09-16T02:07:52.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7feca9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f30e40 0x55a6a7f31220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7feca9000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f3bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7feca9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f30e40 0x55a6a7f31220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:33984/0 (socket says 172.21.15.103:33984) 2024-09-16T02:07:52.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7feca9a00640 1 -- 172.21.15.103:0/1874838888 learned_addr learned my addr 172.21.15.103:0/1874838888 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:52.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7fec9be00640 1 --2- 172.21.15.103:0/1874838888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f3c020 0x55a6a7f3e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7feca9a00640 1 -- 172.21.15.103:0/1874838888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f3c020 msgr2=0x55a6a7f3e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7feca9a00640 1 --2- 172.21.15.103:0/1874838888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f3c020 0x55a6a7f3e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7feca9a00640 1 -- 172.21.15.103:0/1874838888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 msgr2=0x55a6a7f3bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.348+0000 7feca9a00640 1 --2- 172.21.15.103:0/1874838888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f3bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.349+0000 7feca9a00640 1 -- 172.21.15.103:0/1874838888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a7e5dae0 con 0x55a6a7f30e40 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.349+0000 7feca9a00640 1 --2- 172.21.15.103:0/1874838888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f30e40 0x55a6a7f31220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009ca0 tx=0x7fec9c005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=773244e92dcf37c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.349+0000 7fec9a000640 1 -- 172.21.15.103:0/1874838888 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c048070 con 0x55a6a7f30e40 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.349+0000 7fec9a000640 1 -- 172.21.15.103:0/1874838888 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec9c003e50 con 0x55a6a7f30e40 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.349+0000 7fec9a000640 1 -- 172.21.15.103:0/1874838888 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c034040 con 0x55a6a7f30e40 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.349+0000 7feca2000640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fec880a3240 0x7fec880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.349+0000 7feca2000640 1 -- 172.21.15.103:0/588507556 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fec940cfca0 con 0x7feca40745c0 2024-09-16T02:07:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7feca9000640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fec880a3240 0x7fec880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7fecae4d7740 1 -- 172.21.15.103:0/1874838888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f30e40 msgr2=0x55a6a7f31220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7fecae4d7740 1 --2- 172.21.15.103:0/1874838888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f30e40 0x55a6a7f31220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009ca0 tx=0x7fec9c005f80 comp rx=0 tx=0).stop 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7fecae4d7740 1 -- 172.21.15.103:0/1874838888 shutdown_connections 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7fecae4d7740 1 --2- 172.21.15.103:0/1874838888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f3c020 0x55a6a7f3e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7fecae4d7740 1 --2- 172.21.15.103:0/1874838888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f3bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7fecae4d7740 1 --2- 172.21.15.103:0/1874838888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f30e40 0x55a6a7f31220 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7fecae4d7740 1 -- 172.21.15.103:0/1874838888 >> 172.21.15.103:0/1874838888 conn(0x55a6a7f2d3d0 msgr2=0x55a6a7f2d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7fecae4d7740 1 -- 172.21.15.103:0/1874838888 shutdown_connections 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7fecae4d7740 1 -- 172.21.15.103:0/1874838888 wait complete. 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.350+0000 7feca9000640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fec880a3240 0x7fec880a5700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fec940a4920 tx=0x7fec940d36e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.351+0000 7fecae4d7740 1 -- start start 2024-09-16T02:07:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.351+0000 7fecae4d7740 1 Processor -- start 2024-09-16T02:07:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.351+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f30e40 0x55a6a7f38f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.351+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f31760 0x55a6a7f39440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.351+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f3c020 0x55a6a7f39980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.351+0000 7fecae4d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a6a7f3ebf0 con 0x55a6a7f30e40 2024-09-16T02:07:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.351+0000 7fecae4d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6a7e520f0 con 0x55a6a7f3c020 2024-09-16T02:07:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.351+0000 7fecae4d7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a6a7f3ed70 con 0x55a6a7f31760 2024-09-16T02:07:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.351+0000 7fecae4d7740 10 client.0 _ll_register_callbacks cb 0x55a6a7e39bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7fec9be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f31760 0x55a6a7f39440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7fec9be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f31760 0x55a6a7f39440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:33992/0 (socket says 172.21.15.103:33992) 2024-09-16T02:07:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7fec9be00640 1 -- 172.21.15.103:0/1544812272 learned_addr learned my addr 172.21.15.103:0/1544812272 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7feca9a00640 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f30e40 0x55a6a7f38f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7feca9a00640 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f3c020 0x55a6a7f39980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7fec9be00640 1 -- 172.21.15.103:0/1544812272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f3c020 msgr2=0x55a6a7f39980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7fec9be00640 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f3c020 0x55a6a7f39980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7fec9be00640 1 -- 172.21.15.103:0/1544812272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f30e40 msgr2=0x55a6a7f38f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7fec9be00640 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f30e40 0x55a6a7f38f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7fec9be00640 1 -- 172.21.15.103:0/1544812272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6a7c083c0 con 0x55a6a7f31760 2024-09-16T02:07:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7feca9a00640 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f30e40 0x55a6a7f38f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7fec9be00640 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f31760 0x55a6a7f39440 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec8c012730 tx=0x7fec8c012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8496d442023974a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.352+0000 7fec92a00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-16T02:07:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.353+0000 7fec92a00640 1 -- 172.21.15.103:0/1544812272 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c004050 con 0x55a6a7f31760 2024-09-16T02:07:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.353+0000 7fecae4d7740 1 -- 172.21.15.103:0/1544812272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec9c003e50 con 0x55a6a7f31760 2024-09-16T02:07:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.353+0000 7fec92a00640 1 -- 172.21.15.103:0/1544812272 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fec8c0041f0 con 0x55a6a7f31760 2024-09-16T02:07:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.353+0000 7fec92a00640 1 -- 172.21.15.103:0/1544812272 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fec8c021030 con 0x55a6a7f31760 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.353+0000 7fecae4d7740 10 client.25477 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.353+0000 7fecae4d7740 1 -- 172.21.15.103:0/1544812272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a6a7f4ace0 con 0x55a6a7f31760 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.353+0000 7fec92a00640 1 -- 172.21.15.103:0/1544812272 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec9c003e50 con 0x55a6a7f31760 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.354+0000 7fec92a00640 1 -- 172.21.15.103:0/1544812272 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fec8c003990 con 0x55a6a7f31760 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.354+0000 7fecae4d7740 10 client.25477 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.354+0000 7fecae4d7740 10 client.25477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.354+0000 7fecae4d7740 20 client.25477 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.354+0000 7fecae4d7740 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55a6a7f4c0f0 0x55a6a7f4e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.354+0000 7fecae4d7740 4 client.25477 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.354+0000 7fecae4d7740 1 -- 172.21.15.103:0/1544812272 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6a7ea4950 con 0x55a6a7f4c0f0 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.354+0000 7feca9a00640 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55a6a7f4c0f0 0x55a6a7f4e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.355+0000 7feca9a00640 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55a6a7f4c0f0 0x55a6a7f4e550 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8718e6db4b57226c server_cookie=41eb9ff6a0f8aaf8 in_seq=0 out_seq=0 2024-09-16T02:07:52.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.355+0000 7fec92a00640 10 client.25477 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:52.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.355+0000 7fec92a00640 1 -- 172.21.15.103:0/1544812272 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6a7ea4950 con 0x55a6a7f4c0f0 2024-09-16T02:07:52.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.355+0000 7fec92a00640 10 client.25477 handle_command_reply: tid=0 2024-09-16T02:07:52.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 client.25477 shutdown 2024-09-16T02:07:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 -- 172.21.15.103:0/1544812272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f31760 msgr2=0x55a6a7f39440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f31760 0x55a6a7f39440 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec8c012730 tx=0x7fec8c012bf0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 -- 172.21.15.103:0/1544812272 shutdown_connections 2024-09-16T02:07:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55a6a7f4c0f0 0x55a6a7f4e550 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f3c020 0x55a6a7f39980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f31760 0x55a6a7f39440 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 --2- 172.21.15.103:0/1544812272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f30e40 0x55a6a7f38f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 -- 172.21.15.103:0/1544812272 >> 172.21.15.103:0/1544812272 conn(0x55a6a7f2d3d0 msgr2=0x55a6a7f33100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fec92a00640 0 client.25477 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:52.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 -- 172.21.15.103:0/1544812272 shutdown_connections 2024-09-16T02:07:52.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 1 -- 172.21.15.103:0/1544812272 wait complete. 2024-09-16T02:07:52.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.356+0000 7fecae4d7740 20 client.25477 trim_cache size 0 max 16384 2024-09-16T02:07:52.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.369+0000 7fecae4d7740 1 Processor -- start 2024-09-16T02:07:52.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7fecae4d7740 1 -- start start 2024-09-16T02:07:52.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f30e40 0x55a6a7f338a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f3ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f3c020 0x55a6a7f53e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7fecae4d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feca41b2060 con 0x55a6a7f31760 2024-09-16T02:07:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7fecae4d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feca41af930 con 0x55a6a7f30e40 2024-09-16T02:07:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7fecae4d7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feca41ad780 con 0x55a6a7f3c020 2024-09-16T02:07:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7fec9be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f3ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7fec9be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f3ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53586/0 (socket says 172.21.15.103:53586) 2024-09-16T02:07:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7fec9be00640 1 -- 172.21.15.103:0/2156483033 learned_addr learned my addr 172.21.15.103:0/2156483033 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7feca9a00640 1 --2- 172.21.15.103:0/2156483033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f30e40 0x55a6a7f338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.370+0000 7feca9a00640 1 --2- 172.21.15.103:0/2156483033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f3c020 0x55a6a7f53e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.371+0000 7fec9be00640 1 -- 172.21.15.103:0/2156483033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f3c020 msgr2=0x55a6a7f53e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:52.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.371+0000 7fec9be00640 1 --2- 172.21.15.103:0/2156483033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f3c020 0x55a6a7f53e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.371+0000 7fec9be00640 1 -- 172.21.15.103:0/2156483033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f30e40 msgr2=0x55a6a7f338a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:52.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.371+0000 7fec9be00640 1 --2- 172.21.15.103:0/2156483033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f30e40 0x55a6a7f338a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.371+0000 7fec9be00640 1 -- 172.21.15.103:0/2156483033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a7f2d200 con 0x55a6a7f31760 2024-09-16T02:07:52.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.371+0000 7fec9be00640 1 --2- 172.21.15.103:0/2156483033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f3ae70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fec8c026690 tx=0x7fec8c0266c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37421d6bce2b8133 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:52.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.371+0000 7fec99600640 1 -- 172.21.15.103:0/2156483033 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0050e0 con 0x55a6a7f31760 2024-09-16T02:07:52.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.371+0000 7fec99600640 1 -- 172.21.15.103:0/2156483033 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec8c00f920 con 0x55a6a7f31760 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.371+0000 7fec99600640 1 -- 172.21.15.103:0/2156483033 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0035f0 con 0x55a6a7f31760 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.372+0000 7fecae4d7740 1 -- 172.21.15.103:0/2156483033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 msgr2=0x55a6a7f3ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.372+0000 7fecae4d7740 1 --2- 172.21.15.103:0/2156483033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f3ae70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fec8c026690 tx=0x7fec8c0266c0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.372+0000 7fecae4d7740 1 -- 172.21.15.103:0/2156483033 shutdown_connections 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.372+0000 7fecae4d7740 1 --2- 172.21.15.103:0/2156483033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f3c020 0x55a6a7f53e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.372+0000 7fecae4d7740 1 --2- 172.21.15.103:0/2156483033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f3ae70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.372+0000 7fecae4d7740 1 --2- 172.21.15.103:0/2156483033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f30e40 0x55a6a7f338a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.372+0000 7fecae4d7740 1 -- 172.21.15.103:0/2156483033 >> 172.21.15.103:0/2156483033 conn(0x55a6a7f2d3d0 msgr2=0x55a6a7f3a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.372+0000 7fecae4d7740 1 -- 172.21.15.103:0/2156483033 shutdown_connections 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.372+0000 7fecae4d7740 1 -- 172.21.15.103:0/2156483033 wait complete. 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.372+0000 7fecae4d7740 1 -- start start 2024-09-16T02:07:52.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fecae4d7740 1 Processor -- start 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f30e40 0x55a6a7f3fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f40370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fecae4d7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f3c020 0x55a6a7f408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fecae4d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feca41addf0 con 0x55a6a7f31760 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fecae4d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feca41af260 con 0x55a6a7f30e40 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fecae4d7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feca41ae4c0 con 0x55a6a7f3c020 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fecae4d7740 10 client.0 _ll_register_callbacks cb 0x7fec8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fec9be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f40370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7feca9a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f30e40 0x55a6a7f3fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fec9be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f40370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53602/0 (socket says 172.21.15.103:53602) 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.373+0000 7fec9be00640 1 -- 172.21.15.103:0/619023097 learned_addr learned my addr 172.21.15.103:0/619023097 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7feca9a00640 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f3c020 0x55a6a7f408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7fec9be00640 1 -- 172.21.15.103:0/619023097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f3c020 msgr2=0x55a6a7f408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7fec9be00640 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f3c020 0x55a6a7f408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7fec9be00640 1 -- 172.21.15.103:0/619023097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f30e40 msgr2=0x55a6a7f3fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7fec9be00640 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f30e40 0x55a6a7f3fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7fec9be00640 1 -- 172.21.15.103:0/619023097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6a7fb5f90 con 0x55a6a7f31760 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7fec9be00640 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f40370 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fec8c012420 tx=0x7fec8c009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215d3b235a36a6a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7fec92a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7fec92a00640 1 -- 172.21.15.103:0/619023097 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0052d0 con 0x55a6a7f31760 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7fec92a00640 1 -- 172.21.15.103:0/619023097 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fec8c022040 con 0x55a6a7f31760 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.374+0000 7fecae4d7740 1 -- 172.21.15.103:0/619023097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec8c00f920 con 0x55a6a7f31760 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fec92a00640 1 -- 172.21.15.103:0/619023097 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fec8c005470 con 0x55a6a7f31760 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fecae4d7740 10 client.15531 fetch_fsmap learned FSMap version 24 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fecae4d7740 1 -- 172.21.15.103:0/619023097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a6a7fb7210 con 0x55a6a7f31760 2024-09-16T02:07:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fec92a00640 1 -- 172.21.15.103:0/619023097 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec8c00f920 con 0x55a6a7f31760 2024-09-16T02:07:52.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fec92a00640 1 -- 172.21.15.103:0/619023097 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fec8c026b70 con 0x55a6a7f31760 2024-09-16T02:07:52.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fecae4d7740 10 client.15531 fetch_fsmap finished waiting for FSMap version 24 2024-09-16T02:07:52.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fecae4d7740 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T02:07:52.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fecae4d7740 20 client.15531 populate_metadata read hostname 'smithi103' 2024-09-16T02:07:52.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fecae4d7740 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55a6a7fb7970 0x55a6a7fb9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:52.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fecae4d7740 4 client.15531 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T02:07:52.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7fecae4d7740 1 -- 172.21.15.103:0/619023097 --> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6a7e9ee60 con 0x55a6a7fb7970 2024-09-16T02:07:52.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.375+0000 7feca9a00640 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55a6a7fb7970 0x55a6a7fb9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:52.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7feca9a00640 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55a6a7fb7970 0x55a6a7fb9d90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b74741d87160d6f server_cookie=acb1275d125cb636 in_seq=0 out_seq=0 2024-09-16T02:07:52.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fec92a00640 10 client.15531 ms_handle_connect on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:52.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fec92a00640 1 -- 172.21.15.103:0/619023097 <== mds.0 v2:172.21.15.136:6838/3041647128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6a7e9ee60 con 0x55a6a7fb7970 2024-09-16T02:07:52.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fec92a00640 10 client.15531 handle_command_reply: tid=0 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fecae4d7740 1 client.15531 shutdown 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fecae4d7740 1 -- 172.21.15.103:0/619023097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 msgr2=0x55a6a7f40370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fecae4d7740 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f40370 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fec8c012420 tx=0x7fec8c009330 comp rx=0 tx=0).stop 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fecae4d7740 1 -- 172.21.15.103:0/619023097 shutdown_connections 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fecae4d7740 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.136:6838/3041647128,v1:172.21.15.136:6839/3041647128] conn(0x55a6a7fb7970 0x55a6a7fb9d90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fecae4d7740 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a6a7f3c020 0x55a6a7f408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fecae4d7740 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6a7f31760 0x55a6a7f40370 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fecae4d7740 1 --2- 172.21.15.103:0/619023097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6a7f30e40 0x55a6a7f3fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fecae4d7740 1 -- 172.21.15.103:0/619023097 >> 172.21.15.103:0/619023097 conn(0x55a6a7f2d3d0 msgr2=0x55a6a7f396e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.376+0000 7fec92a00640 0 client.15531 ms_handle_reset on v2:172.21.15.136:6838/3041647128 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.377+0000 7fecae4d7740 1 -- 172.21.15.103:0/619023097 shutdown_connections 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.377+0000 7fecae4d7740 1 -- 172.21.15.103:0/619023097 wait complete. 2024-09-16T02:07:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.377+0000 7fecae4d7740 20 client.15531 trim_cache size 0 max 16384 2024-09-16T02:07:52.384 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.377+0000 7fecaa400640 1 -- 172.21.15.103:0/588507556 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fec880a3240 msgr2=0x7fec880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.377+0000 7fecaa400640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fec880a3240 0x7fec880a5700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fec940a4920 tx=0x7fec940d36e0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.377+0000 7fecaa400640 1 -- 172.21.15.103:0/588507556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca40745c0 msgr2=0x7feca419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.377+0000 7fecaa400640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca40745c0 0x7feca419a440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fec9400ea60 tx=0x7fec9400ef20 comp rx=0 tx=0).stop 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.378+0000 7fecaa400640 1 -- 172.21.15.103:0/588507556 shutdown_connections 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.378+0000 7fecaa400640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fec880a3240 0x7fec880a5700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.378+0000 7fecaa400640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca410e260 0x7feca419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.378+0000 7fecaa400640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca4074ee0 0x7feca419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.378+0000 7fecaa400640 1 --2- 172.21.15.103:0/588507556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca40745c0 0x7feca419a440 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.378+0000 7fecaa400640 1 -- 172.21.15.103:0/588507556 >> 172.21.15.103:0/588507556 conn(0x7feca40fd6e0 msgr2=0x7feca40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.378+0000 7fecaa400640 1 -- 172.21.15.103:0/588507556 shutdown_connections 2024-09-16T02:07:52.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:52.378+0000 7fecaa400640 1 -- 172.21.15.103:0/588507556 wait complete. 2024-09-16T02:07:53.222 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-16T02:07:53.223 INFO:tasks.fwd_scrub:done joining 2024-09-16T02:07:53.224 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-16T02:07:53.234 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T02:07:53.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.389+0000 7f4bef600640 1 -- 172.21.15.103:0/377168882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4be810e100 msgr2=0x7f4be81104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:53.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.389+0000 7f4bef600640 1 --2- 172.21.15.103:0/377168882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4be810e100 0x7f4be81104f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009ca0 tx=0x7f4be002a8c0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.389+0000 7f4bef600640 1 -- 172.21.15.103:0/377168882 shutdown_connections 2024-09-16T02:07:53.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.389+0000 7f4bef600640 1 --2- 172.21.15.103:0/377168882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4be810e100 0x7f4be81104f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.389+0000 7f4bef600640 1 --2- 172.21.15.103:0/377168882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be8074c50 0x7f4be80750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.389+0000 7f4bef600640 1 --2- 172.21.15.103:0/377168882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8076230 0x7f4be8074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.389+0000 7f4bef600640 1 -- 172.21.15.103:0/377168882 >> 172.21.15.103:0/377168882 conn(0x7f4be80fd2b0 msgr2=0x7f4be80ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:53.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.389+0000 7f4bef600640 1 -- 172.21.15.103:0/377168882 shutdown_connections 2024-09-16T02:07:53.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.389+0000 7f4bef600640 1 -- 172.21.15.103:0/377168882 wait complete. 2024-09-16T02:07:53.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.390+0000 7f4bef600640 1 Processor -- start 2024-09-16T02:07:53.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.390+0000 7f4bef600640 1 -- start start 2024-09-16T02:07:53.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.390+0000 7f4bef600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8074c50 0x7f4be819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:53.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.390+0000 7f4bef600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4be8076230 0x7f4be819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:53.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.390+0000 7f4bef600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be810e100 0x7f4be81a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:53.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.390+0000 7f4bef600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8113dc0 con 0x7f4be8076230 2024-09-16T02:07:53.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.390+0000 7f4bef600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8113c40 con 0x7f4be810e100 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.390+0000 7f4bef600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8113f40 con 0x7f4be8074c50 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4be8076230 0x7f4be819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4be8076230 0x7f4be819e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53618/0 (socket says 172.21.15.103:53618) 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be7e00640 1 -- 172.21.15.103:0/3528556546 learned_addr learned my addr 172.21.15.103:0/3528556546 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4bed600640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be810e100 0x7f4be81a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4becc00640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8074c50 0x7f4be819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be7e00640 1 -- 172.21.15.103:0/3528556546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8074c50 msgr2=0x7f4be819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be7e00640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8074c50 0x7f4be819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be7e00640 1 -- 172.21.15.103:0/3528556546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be810e100 msgr2=0x7f4be81a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be7e00640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be810e100 0x7f4be81a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be7e00640 1 -- 172.21.15.103:0/3528556546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be81a6020 con 0x7f4be8076230 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be7e00640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4be8076230 0x7f4be819e940 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400d950 tx=0x7f4bd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be5600640 1 -- 172.21.15.103:0/3528556546 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4017030 con 0x7f4be8076230 2024-09-16T02:07:53.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.391+0000 7f4be5600640 1 -- 172.21.15.103:0/3528556546 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd4004020 con 0x7f4be8076230 2024-09-16T02:07:53.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.392+0000 7f4bef600640 1 -- 172.21.15.103:0/3528556546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be81a6310 con 0x7f4be8076230 2024-09-16T02:07:53.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.392+0000 7f4be5600640 1 -- 172.21.15.103:0/3528556546 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40047a0 con 0x7f4be8076230 2024-09-16T02:07:53.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.392+0000 7f4bef600640 1 -- 172.21.15.103:0/3528556546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be81a6860 con 0x7f4be8076230 2024-09-16T02:07:53.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.393+0000 7f4bef600640 1 -- 172.21.15.103:0/3528556546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be81042d0 con 0x7f4be8076230 2024-09-16T02:07:53.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.394+0000 7f4be5600640 1 -- 172.21.15.103:0/3528556546 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4bd40041d0 con 0x7f4be8076230 2024-09-16T02:07:53.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.396+0000 7f4be5600640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4bd00a3250 0x7f4bd00a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:53.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.396+0000 7f4be5600640 1 -- 172.21.15.103:0/3528556546 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f4bd40cf3b0 con 0x7f4be8076230 2024-09-16T02:07:53.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.396+0000 7f4becc00640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4bd00a3250 0x7f4bd00a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:53.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.397+0000 7f4becc00640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4bd00a3250 0x7f4bd00a5710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009b50 tx=0x7f4bdc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:53.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.400+0000 7f4be5600640 1 -- 172.21.15.103:0/3528556546 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bd409bff0 con 0x7f4be8076230 2024-09-16T02:07:53.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.577+0000 7f4bef600640 1 -- 172.21.15.103:0/3528556546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4be8062df0 con 0x7f4be8076230 2024-09-16T02:07:53.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.579+0000 7f4be5600640 1 -- 172.21.15.103:0/3528556546 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23048 (secure 0 0 0) 0x7f4bd40860c0 con 0x7f4be8076230 2024-09-16T02:07:53.581 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:07:53.581 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":81,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:05:30.540044+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":10,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":7,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:07:53.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 -- 172.21.15.103:0/3528556546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4bd00a3250 msgr2=0x7f4bd00a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:53.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4bd00a3250 0x7f4bd00a5710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009b50 tx=0x7f4bdc005f90 comp rx=0 tx=0).stop 2024-09-16T02:07:53.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 -- 172.21.15.103:0/3528556546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4be8076230 msgr2=0x7f4be819e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:53.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4be8076230 0x7f4be819e940 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400d950 tx=0x7f4bd400de10 comp rx=0 tx=0).stop 2024-09-16T02:07:53.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 -- 172.21.15.103:0/3528556546 shutdown_connections 2024-09-16T02:07:53.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f4bd00a3250 0x7f4bd00a5710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be810e100 0x7f4be81a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4be8076230 0x7f4be819e940 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 --2- 172.21.15.103:0/3528556546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8074c50 0x7f4be819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 -- 172.21.15.103:0/3528556546 >> 172.21.15.103:0/3528556546 conn(0x7f4be80fd2b0 msgr2=0x7f4be81014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:53.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 -- 172.21.15.103:0/3528556546 shutdown_connections 2024-09-16T02:07:53.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.581+0000 7f4bef600640 1 -- 172.21.15.103:0/3528556546 wait complete. 2024-09-16T02:07:53.600 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T01:58:50.953500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T02:03:33.662251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T02:03:36.012448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T02:07:53.600 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T02:07:53.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.759+0000 7faf30000640 1 -- 172.21.15.103:0/2651664177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf28103d30 msgr2=0x7faf28104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:53.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.759+0000 7faf30000640 1 --2- 172.21.15.103:0/2651664177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf28103d30 0x7faf28104130 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009860 tx=0x7faf1c02a340 comp rx=0 tx=0).stop 2024-09-16T02:07:53.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.759+0000 7faf30000640 1 -- 172.21.15.103:0/2651664177 shutdown_connections 2024-09-16T02:07:53.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.759+0000 7faf30000640 1 --2- 172.21.15.103:0/2651664177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf281058e0 0x7faf2810c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.759+0000 7faf30000640 1 --2- 172.21.15.103:0/2651664177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf28104f20 0x7faf281053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.759+0000 7faf30000640 1 --2- 172.21.15.103:0/2651664177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf28103d30 0x7faf28104130 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.759+0000 7faf30000640 1 -- 172.21.15.103:0/2651664177 >> 172.21.15.103:0/2651664177 conn(0x7faf280ff470 msgr2=0x7faf281018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:53.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.759+0000 7faf30000640 1 -- 172.21.15.103:0/2651664177 shutdown_connections 2024-09-16T02:07:53.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.759+0000 7faf30000640 1 -- 172.21.15.103:0/2651664177 wait complete. 2024-09-16T02:07:53.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.760+0000 7faf30000640 1 Processor -- start 2024-09-16T02:07:53.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.760+0000 7faf30000640 1 -- start start 2024-09-16T02:07:53.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.760+0000 7faf30000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf28103d30 0x7faf2819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:53.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.760+0000 7faf30000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf28104f20 0x7faf2819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:53.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.760+0000 7faf30000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf281058e0 0x7faf281a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:53.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf30000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faf2810f990 con 0x7faf281058e0 2024-09-16T02:07:53.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf30000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf2810f810 con 0x7faf28104f20 2024-09-16T02:07:53.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf30000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faf2810fb10 con 0x7faf28103d30 2024-09-16T02:07:53.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf2e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf281058e0 0x7faf281a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf2e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf281058e0 0x7faf281a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53646/0 (socket says 172.21.15.103:53646) 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf2e000640 1 -- 172.21.15.103:0/3180493696 learned_addr learned my addr 172.21.15.103:0/3180493696 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf2d600640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf28103d30 0x7faf2819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf2cc00640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf28104f20 0x7faf2819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf2e000640 1 -- 172.21.15.103:0/3180493696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf28103d30 msgr2=0x7faf2819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf2e000640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf28103d30 0x7faf2819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf2e000640 1 -- 172.21.15.103:0/3180493696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf28104f20 msgr2=0x7faf2819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf2e000640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf28104f20 0x7faf2819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.761+0000 7faf2e000640 1 -- 172.21.15.103:0/3180493696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf281a1ca0 con 0x7faf281058e0 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.762+0000 7faf2e000640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf281058e0 0x7faf281a1630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faf2000eb10 tx=0x7faf2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.762+0000 7faf26000640 1 -- 172.21.15.103:0/3180493696 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20004b20 con 0x7faf281058e0 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.762+0000 7faf26000640 1 -- 172.21.15.103:0/3180493696 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf20004cc0 con 0x7faf281058e0 2024-09-16T02:07:53.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.762+0000 7faf26000640 1 -- 172.21.15.103:0/3180493696 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20010760 con 0x7faf281058e0 2024-09-16T02:07:53.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.762+0000 7faf30000640 1 -- 172.21.15.103:0/3180493696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf281a1f90 con 0x7faf281058e0 2024-09-16T02:07:53.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.762+0000 7faf30000640 1 -- 172.21.15.103:0/3180493696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf281a2400 con 0x7faf281058e0 2024-09-16T02:07:53.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.763+0000 7faf30000640 1 -- 172.21.15.103:0/3180493696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf2810b1b0 con 0x7faf281058e0 2024-09-16T02:07:53.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.765+0000 7faf26000640 1 -- 172.21.15.103:0/3180493696 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7faf20010900 con 0x7faf281058e0 2024-09-16T02:07:53.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.766+0000 7faf26000640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faf100a3250 0x7faf100a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:53.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.766+0000 7faf26000640 1 -- 172.21.15.103:0/3180493696 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7faf200d0110 con 0x7faf281058e0 2024-09-16T02:07:53.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.766+0000 7faf2d600640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faf100a3250 0x7faf100a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:53.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.767+0000 7faf2d600640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faf100a3250 0x7faf100a5710 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009830 tx=0x7faf1c030570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:53.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.768+0000 7faf26000640 1 -- 172.21.15.103:0/3180493696 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf200541b0 con 0x7faf281058e0 2024-09-16T02:07:53.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.941+0000 7faf30000640 1 -- 172.21.15.103:0/3180493696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf2802da10 con 0x7faf281058e0 2024-09-16T02:07:53.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.941+0000 7faf26000640 1 -- 172.21.15.103:0/3180493696 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7faf2009cdb0 con 0x7faf281058e0 2024-09-16T02:07:53.943 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.943+0000 7faf30000640 1 -- 172.21.15.103:0/3180493696 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faf100a3250 msgr2=0x7faf100a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.943+0000 7faf30000640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faf100a3250 0x7faf100a5710 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009830 tx=0x7faf1c030570 comp rx=0 tx=0).stop 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.943+0000 7faf30000640 1 -- 172.21.15.103:0/3180493696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf281058e0 msgr2=0x7faf281a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.943+0000 7faf30000640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf281058e0 0x7faf281a1630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faf2000eb10 tx=0x7faf2000c2d0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.944+0000 7faf30000640 1 -- 172.21.15.103:0/3180493696 shutdown_connections 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.944+0000 7faf30000640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7faf100a3250 0x7faf100a5710 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.944+0000 7faf30000640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf281058e0 0x7faf281a1630 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.944+0000 7faf30000640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf28104f20 0x7faf2819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.944+0000 7faf30000640 1 --2- 172.21.15.103:0/3180493696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf28103d30 0x7faf2819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.944+0000 7faf30000640 1 -- 172.21.15.103:0/3180493696 >> 172.21.15.103:0/3180493696 conn(0x7faf280ff470 msgr2=0x7faf28108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.944+0000 7faf30000640 1 -- 172.21.15.103:0/3180493696 shutdown_connections 2024-09-16T02:07:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:53.944+0000 7faf30000640 1 -- 172.21.15.103:0/3180493696 wait complete. 2024-09-16T02:07:53.959 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:53 smithi103 ceph-mon[30398]: pgmap v413: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T02:07:53.960 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:53 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3528556546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:07:53.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T02:07:54.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:53 smithi136 ceph-mon[35323]: pgmap v413: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T02:07:54.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:53 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3528556546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:07:54.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:53 smithi172 ceph-mon[35340]: pgmap v413: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-16T02:07:54.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:53 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3528556546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:07:54.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.117+0000 7f6b32600640 1 -- 172.21.15.103:0/2269195340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b2c078790 msgr2=0x7f6b2c079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.117+0000 7f6b32600640 1 --2- 172.21.15.103:0/2269195340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b2c078790 0x7f6b2c079030 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009ca0 tx=0x7f6b2402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.117+0000 7f6b32600640 1 -- 172.21.15.103:0/2269195340 shutdown_connections 2024-09-16T02:07:54.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.117+0000 7f6b32600640 1 --2- 172.21.15.103:0/2269195340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b2c078790 0x7f6b2c079030 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.117+0000 7f6b32600640 1 --2- 172.21.15.103:0/2269195340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b2c077df0 0x7f6b2c078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.117+0000 7f6b32600640 1 --2- 172.21.15.103:0/2269195340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c076bb0 0x7f6b2c076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.117+0000 7f6b32600640 1 -- 172.21.15.103:0/2269195340 >> 172.21.15.103:0/2269195340 conn(0x7f6b2c0ff4a0 msgr2=0x7f6b2c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:54.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.117+0000 7f6b32600640 1 -- 172.21.15.103:0/2269195340 shutdown_connections 2024-09-16T02:07:54.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.117+0000 7f6b32600640 1 -- 172.21.15.103:0/2269195340 wait complete. 2024-09-16T02:07:54.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.118+0000 7f6b32600640 1 Processor -- start 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.118+0000 7f6b32600640 1 -- start start 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b32600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b2c076bb0 0x7f6b2c19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b32600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b2c077df0 0x7f6b2c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b32600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c078790 0x7f6b2c1a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b32600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c113e00 con 0x7f6b2c077df0 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b32600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c113c80 con 0x7f6b2c076bb0 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b32600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c113f80 con 0x7f6b2c078790 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b22a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b2c077df0 0x7f6b2c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b22a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b2c077df0 0x7f6b2c19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53662/0 (socket says 172.21.15.103:53662) 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b22a00640 1 -- 172.21.15.103:0/914169749 learned_addr learned my addr 172.21.15.103:0/914169749 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b2b400640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b2c076bb0 0x7f6b2c19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b2be00640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c078790 0x7f6b2c1a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b22a00640 1 -- 172.21.15.103:0/914169749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c078790 msgr2=0x7f6b2c1a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b22a00640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c078790 0x7f6b2c1a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b22a00640 1 -- 172.21.15.103:0/914169749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b2c076bb0 msgr2=0x7f6b2c19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b22a00640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b2c076bb0 0x7f6b2c19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.119+0000 7f6b22a00640 1 -- 172.21.15.103:0/914169749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c1a6060 con 0x7f6b2c077df0 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.120+0000 7f6b22a00640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b2c077df0 0x7f6b2c19e980 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400d950 tx=0x7f6b1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.120+0000 7f6b28c00640 1 -- 172.21.15.103:0/914169749 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14017030 con 0x7f6b2c077df0 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.120+0000 7f6b28c00640 1 -- 172.21.15.103:0/914169749 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b14004020 con 0x7f6b2c077df0 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.120+0000 7f6b32600640 1 -- 172.21.15.103:0/914169749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c1a6350 con 0x7f6b2c077df0 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.120+0000 7f6b28c00640 1 -- 172.21.15.103:0/914169749 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b140047a0 con 0x7f6b2c077df0 2024-09-16T02:07:54.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.120+0000 7f6b32600640 1 -- 172.21.15.103:0/914169749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c1a68a0 con 0x7f6b2c077df0 2024-09-16T02:07:54.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.121+0000 7f6b32600640 1 -- 172.21.15.103:0/914169749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c110880 con 0x7f6b2c077df0 2024-09-16T02:07:54.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.124+0000 7f6b28c00640 1 -- 172.21.15.103:0/914169749 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6b140041d0 con 0x7f6b2c077df0 2024-09-16T02:07:54.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.126+0000 7f6b28c00640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6b100a3250 0x7f6b100a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.126+0000 7f6b28c00640 1 -- 172.21.15.103:0/914169749 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f6b140cfa40 con 0x7f6b2c077df0 2024-09-16T02:07:54.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.126+0000 7f6b2b400640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6b100a3250 0x7f6b100a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.127+0000 7f6b2b400640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6b100a3250 0x7f6b100a5710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009bd0 tx=0x7f6b1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:54.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.128+0000 7f6b28c00640 1 -- 172.21.15.103:0/914169749 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b1409c680 con 0x7f6b2c077df0 2024-09-16T02:07:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.304+0000 7f6b32600640 1 -- 172.21.15.103:0/914169749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6b2c02da10 con 0x7f6b2c077df0 2024-09-16T02:07:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.304+0000 7f6b28c00640 1 -- 172.21.15.103:0/914169749 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7f6b1409bf80 con 0x7f6b2c077df0 2024-09-16T02:07:54.307 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-09-16T02:07:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.306+0000 7f6b32600640 1 -- 172.21.15.103:0/914169749 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6b100a3250 msgr2=0x7f6b100a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.306+0000 7f6b32600640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6b100a3250 0x7f6b100a5710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009bd0 tx=0x7f6b1c005f90 comp rx=0 tx=0).stop 2024-09-16T02:07:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.306+0000 7f6b32600640 1 -- 172.21.15.103:0/914169749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b2c077df0 msgr2=0x7f6b2c19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.306+0000 7f6b32600640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b2c077df0 0x7f6b2c19e980 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400d950 tx=0x7f6b1400de10 comp rx=0 tx=0).stop 2024-09-16T02:07:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.307+0000 7f6b32600640 1 -- 172.21.15.103:0/914169749 shutdown_connections 2024-09-16T02:07:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.307+0000 7f6b32600640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f6b100a3250 0x7f6b100a5710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.307+0000 7f6b32600640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c078790 0x7f6b2c1a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.307+0000 7f6b32600640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b2c077df0 0x7f6b2c19e980 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.307+0000 7f6b32600640 1 --2- 172.21.15.103:0/914169749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b2c076bb0 0x7f6b2c19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.307+0000 7f6b32600640 1 -- 172.21.15.103:0/914169749 >> 172.21.15.103:0/914169749 conn(0x7f6b2c0ff4a0 msgr2=0x7f6b2c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:54.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.307+0000 7f6b32600640 1 -- 172.21.15.103:0/914169749 shutdown_connections 2024-09-16T02:07:54.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.307+0000 7f6b32600640 1 -- 172.21.15.103:0/914169749 wait complete. 2024-09-16T02:07:54.325 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T02:07:54.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.491+0000 7fea5aa00640 1 -- 172.21.15.103:0/3341602855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea54103d20 msgr2=0x7fea54104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.491+0000 7fea5aa00640 1 --2- 172.21.15.103:0/3341602855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea54103d20 0x7fea54104120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fea44009900 tx=0x7fea4402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.491+0000 7fea5aa00640 1 -- 172.21.15.103:0/3341602855 shutdown_connections 2024-09-16T02:07:54.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.491+0000 7fea5aa00640 1 --2- 172.21.15.103:0/3341602855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea541058d0 0x7fea5410c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.491+0000 7fea5aa00640 1 --2- 172.21.15.103:0/3341602855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea54104f10 0x7fea54105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.491+0000 7fea5aa00640 1 --2- 172.21.15.103:0/3341602855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea54103d20 0x7fea54104120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.491+0000 7fea5aa00640 1 -- 172.21.15.103:0/3341602855 >> 172.21.15.103:0/3341602855 conn(0x7fea540ff4a0 msgr2=0x7fea541018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:54.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.491+0000 7fea5aa00640 1 -- 172.21.15.103:0/3341602855 shutdown_connections 2024-09-16T02:07:54.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.491+0000 7fea5aa00640 1 -- 172.21.15.103:0/3341602855 wait complete. 2024-09-16T02:07:54.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.492+0000 7fea5aa00640 1 Processor -- start 2024-09-16T02:07:54.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.492+0000 7fea5aa00640 1 -- start start 2024-09-16T02:07:54.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.492+0000 7fea5aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea54103d20 0x7fea5419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.492+0000 7fea5aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea54104f10 0x7fea5419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.492+0000 7fea5aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541058d0 0x7fea541a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.492+0000 7fea5aa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea5410f980 con 0x7fea54103d20 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.492+0000 7fea5aa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea5410f800 con 0x7fea541058d0 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.492+0000 7fea5aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea5410fb00 con 0x7fea54104f10 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea53e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea54103d20 0x7fea5419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea53e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea54103d20 0x7fea5419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53688/0 (socket says 172.21.15.103:53688) 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea53e00640 1 -- 172.21.15.103:0/690723923 learned_addr learned my addr 172.21.15.103:0/690723923 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea53400640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea54104f10 0x7fea5419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea58a00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541058d0 0x7fea541a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea53e00640 1 -- 172.21.15.103:0/690723923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea54104f10 msgr2=0x7fea5419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea53e00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea54104f10 0x7fea5419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea53e00640 1 -- 172.21.15.103:0/690723923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541058d0 msgr2=0x7fea541a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea53e00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541058d0 0x7fea541a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea53e00640 1 -- 172.21.15.103:0/690723923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea541a1cb0 con 0x7fea54103d20 2024-09-16T02:07:54.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea53e00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea54103d20 0x7fea5419a090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fea44031610 tx=0x7fea44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:54.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.493+0000 7fea50c00640 1 -- 172.21.15.103:0/690723923 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4403f030 con 0x7fea54103d20 2024-09-16T02:07:54.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.494+0000 7fea50c00640 1 -- 172.21.15.103:0/690723923 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea44003ad0 con 0x7fea54103d20 2024-09-16T02:07:54.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.494+0000 7fea5aa00640 1 -- 172.21.15.103:0/690723923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea541a1f40 con 0x7fea54103d20 2024-09-16T02:07:54.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.494+0000 7fea50c00640 1 -- 172.21.15.103:0/690723923 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44035070 con 0x7fea54103d20 2024-09-16T02:07:54.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.494+0000 7fea5aa00640 1 -- 172.21.15.103:0/690723923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea541a2350 con 0x7fea54103d20 2024-09-16T02:07:54.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.495+0000 7fea5aa00640 1 -- 172.21.15.103:0/690723923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5410b150 con 0x7fea54103d20 2024-09-16T02:07:54.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.497+0000 7fea50c00640 1 -- 172.21.15.103:0/690723923 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fea44042030 con 0x7fea54103d20 2024-09-16T02:07:54.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.498+0000 7fea50c00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fea380a3250 0x7fea380a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.498+0000 7fea50c00640 1 -- 172.21.15.103:0/690723923 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fea440eff30 con 0x7fea54103d20 2024-09-16T02:07:54.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.498+0000 7fea53400640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fea380a3250 0x7fea380a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.499+0000 7fea53400640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fea380a3250 0x7fea380a5710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fea400095f0 tx=0x7fea40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.501+0000 7fea50c00640 1 -- 172.21.15.103:0/690723923 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea440bcbd0 con 0x7fea54103d20 2024-09-16T02:07:54.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.677+0000 7fea5aa00640 1 -- 172.21.15.103:0/690723923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fea54062f30 con 0x7fea54103d20 2024-09-16T02:07:54.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.677+0000 7fea50c00640 1 -- 172.21.15.103:0/690723923 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7fea440bc4d0 con 0x7fea54103d20 2024-09-16T02:07:54.680 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 64 2024-09-16T02:07:54.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 -- 172.21.15.103:0/690723923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fea380a3250 msgr2=0x7fea380a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fea380a3250 0x7fea380a5710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fea400095f0 tx=0x7fea40005f90 comp rx=0 tx=0).stop 2024-09-16T02:07:54.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 -- 172.21.15.103:0/690723923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea54103d20 msgr2=0x7fea5419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea54103d20 0x7fea5419a090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fea44031610 tx=0x7fea44004270 comp rx=0 tx=0).stop 2024-09-16T02:07:54.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 -- 172.21.15.103:0/690723923 shutdown_connections 2024-09-16T02:07:54.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fea380a3250 0x7fea380a5710 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541058d0 0x7fea541a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea54104f10 0x7fea5419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 --2- 172.21.15.103:0/690723923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea54103d20 0x7fea5419a090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 -- 172.21.15.103:0/690723923 >> 172.21.15.103:0/690723923 conn(0x7fea540ff4a0 msgr2=0x7fea54108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:54.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 -- 172.21.15.103:0/690723923 shutdown_connections 2024-09-16T02:07:54.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.680+0000 7fea5aa00640 1 -- 172.21.15.103:0/690723923 wait complete. 2024-09-16T02:07:54.698 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-16T02:07:54.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.852+0000 7f477d600640 1 -- 172.21.15.103:0/2426453762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47780761d0 msgr2=0x7f4778074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.852+0000 7f477d600640 1 --2- 172.21.15.103:0/2426453762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47780761d0 0x7f4778074620 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4768009900 tx=0x7f476802a480 comp rx=0 tx=0).stop 2024-09-16T02:07:54.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.852+0000 7f477d600640 1 -- 172.21.15.103:0/2426453762 shutdown_connections 2024-09-16T02:07:54.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.852+0000 7f477d600640 1 --2- 172.21.15.103:0/2426453762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47781099c0 0x7f477810be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.852+0000 7f477d600640 1 --2- 172.21.15.103:0/2426453762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4778074bf0 0x7f4778075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.852+0000 7f477d600640 1 --2- 172.21.15.103:0/2426453762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47780761d0 0x7f4778074620 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.852+0000 7f477d600640 1 -- 172.21.15.103:0/2426453762 >> 172.21.15.103:0/2426453762 conn(0x7f47780fd2b0 msgr2=0x7f47780ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:54.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.852+0000 7f477d600640 1 -- 172.21.15.103:0/2426453762 shutdown_connections 2024-09-16T02:07:54.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.853+0000 7f477d600640 1 -- 172.21.15.103:0/2426453762 wait complete. 2024-09-16T02:07:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.853+0000 7f477d600640 1 Processor -- start 2024-09-16T02:07:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.853+0000 7f477d600640 1 -- start start 2024-09-16T02:07:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f477d600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4778074bf0 0x7f477819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f477d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47780761d0 0x7f477819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f477d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47781099c0 0x7f47781a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f477d600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f477810f9c0 con 0x7f47780761d0 2024-09-16T02:07:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f477d600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f477810f840 con 0x7f47781099c0 2024-09-16T02:07:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f477d600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f477810fb40 con 0x7f4778074bf0 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f476e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47780761d0 0x7f477819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f476e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47780761d0 0x7f477819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53706/0 (socket says 172.21.15.103:53706) 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f476e000640 1 -- 172.21.15.103:0/249429857 learned_addr learned my addr 172.21.15.103:0/249429857 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f4777400640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47781099c0 0x7f47781a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f4776a00640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4778074bf0 0x7f477819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f476e000640 1 -- 172.21.15.103:0/249429857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4778074bf0 msgr2=0x7f477819a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f476e000640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4778074bf0 0x7f477819a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f476e000640 1 -- 172.21.15.103:0/249429857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47781099c0 msgr2=0x7f47781a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f476e000640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47781099c0 0x7f47781a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f476e000640 1 -- 172.21.15.103:0/249429857 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47781a1cc0 con 0x7f47780761d0 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.854+0000 7f4776a00640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4778074bf0 0x7f477819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.855+0000 7f476e000640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47780761d0 0x7f477819a5e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f476000e9f0 tx=0x7f476000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.855+0000 7f476fe00640 1 -- 172.21.15.103:0/249429857 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476000cd80 con 0x7f47780761d0 2024-09-16T02:07:54.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.855+0000 7f476fe00640 1 -- 172.21.15.103:0/249429857 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4760005500 con 0x7f47780761d0 2024-09-16T02:07:54.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.855+0000 7f477d600640 1 -- 172.21.15.103:0/249429857 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47781a1fb0 con 0x7f47780761d0 2024-09-16T02:07:54.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.855+0000 7f476fe00640 1 -- 172.21.15.103:0/249429857 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760010690 con 0x7f47780761d0 2024-09-16T02:07:54.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.855+0000 7f477d600640 1 -- 172.21.15.103:0/249429857 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781a2470 con 0x7f47780761d0 2024-09-16T02:07:54.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.856+0000 7f477d600640 1 -- 172.21.15.103:0/249429857 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4778108780 con 0x7f47780761d0 2024-09-16T02:07:54.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.858+0000 7f476fe00640 1 -- 172.21.15.103:0/249429857 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f47600046c0 con 0x7f47780761d0 2024-09-16T02:07:54.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.859+0000 7f476fe00640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f475c0a32d0 0x7f475c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:07:54.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.859+0000 7f476fe00640 1 -- 172.21.15.103:0/249429857 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7f47600cfdf0 con 0x7f47780761d0 2024-09-16T02:07:54.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.860+0000 7f4776a00640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f475c0a32d0 0x7f475c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:07:54.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.860+0000 7f4776a00640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f475c0a32d0 0x7f475c0a5790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4768000c00 tx=0x7f47680095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:07:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:54.863+0000 7f476fe00640 1 -- 172.21.15.103:0/249429857 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f476009ca30 con 0x7f47780761d0 2024-09-16T02:07:55.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:54 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3180493696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:07:55.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:54 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/914169749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:07:55.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:54 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3180493696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:07:55.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:54 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/914169749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:07:55.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:54 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3180493696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T02:07:55.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:54 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/914169749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T02:07:55.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.084+0000 7f477d600640 1 -- 172.21.15.103:0/249429857 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f4778062e90 con 0x7f47780761d0 2024-09-16T02:07:55.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.095+0000 7f476fe00640 1 -- 172.21.15.103:0/249429857 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+742346 (secure 0 0 0) 0x7f476009c330 con 0x7f47780761d0 2024-09-16T02:07:55.108 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:07:55.108 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 20, 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: "active_gid": 14150, 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: "active_name": "x", 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addrs": { 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6800", 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3075264030 2024-09-16T02:07:55.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6801", 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3075264030 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addr": "172.21.15.103:6801/3075264030", 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: "active_change": "2024-09-16T01:56:28.360056+0000", 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: "active_mgr_features": 4540701547738038271, 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: "available": true, 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: "standbys": [ 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.110 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24109, 2024-09-16T02:07:55.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "z", 2024-09-16T02:07:55.111 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540701547738038271, 2024-09-16T02:07:55.111 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-16T02:07:55.111 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-16T02:07:55.111 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.111 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.111 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.111 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:55.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.112 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.113 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.114 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.115 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.117 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-16T02:07:55.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-16T02:07:55.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.123 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.124 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.126 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.127 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T02:07:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-16T02:07:55.131 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.132 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-16T02:07:55.133 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.134 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-16T02:07:55.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.136 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.138 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.140 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-16T02:07:55.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-16T02:07:55.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.142 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.142 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:55.142 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.142 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.142 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.142 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T02:07:55.142 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-16T02:07:55.143 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.144 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.145 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-16T02:07:55.146 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-16T02:07:55.147 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.148 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-16T02:07:55.149 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.149 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.149 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.149 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-16T02:07:55.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-16T02:07:55.149 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.149 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.149 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.151 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.152 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-16T02:07:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-16T02:07:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T02:07:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T02:07:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-16T02:07:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T02:07:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.166 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-16T02:07:55.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.170 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-16T02:07:55.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-16T02:07:55.170 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.170 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.170 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T02:07:55.170 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.171 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-16T02:07:55.172 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T02:07:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.174 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.174 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.175 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.177 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.178 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-16T02:07:55.178 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.178 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.178 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.178 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-16T02:07:55.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-16T02:07:55.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.179 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.179 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.179 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T02:07:55.179 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.179 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.179 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.179 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-16T02:07:55.179 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.180 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-16T02:07:55.181 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.181 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.181 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.181 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-16T02:07:55.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-16T02:07:55.181 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.181 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.181 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.181 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.182 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T02:07:55.183 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-16T02:07:55.184 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.185 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.186 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.188 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.189 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.190 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T02:07:55.191 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.192 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.193 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-16T02:07:55.194 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.197 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.199 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.200 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.202 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.203 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-16T02:07:55.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.205 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.207 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.208 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.209 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.210 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.211 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.212 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-16T02:07:55.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.214 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.215 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.216 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.217 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.217 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.217 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T02:07:55.217 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.217 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.217 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.217 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-16T02:07:55.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.219 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.220 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.222 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.223 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.228 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.228 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.228 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.228 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-16T02:07:55.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-16T02:07:55.228 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.228 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.228 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.229 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.230 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.231 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.232 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.233 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T02:07:55.234 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.235 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T02:07:55.236 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.238 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.239 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.240 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.241 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.242 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.243 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.244 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.245 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.246 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.247 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.250 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.251 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.252 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.253 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.254 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.255 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.256 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T02:07:55.257 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T02:07:55.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T02:07:55.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T02:07:55.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.261 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T02:07:55.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T02:07:55.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.267 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.268 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.269 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.270 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.272 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.273 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T02:07:55.274 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T02:07:55.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.276 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.277 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.278 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.279 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.280 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.281 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.282 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.283 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.283 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.283 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.283 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.283 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.283 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.284 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.284 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.284 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.284 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.284 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.284 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.284 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.284 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.287 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.288 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.289 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.290 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.290 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-16T02:07:55.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-16T02:07:55.290 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.290 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-16T02:07:55.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.292 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.293 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.294 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-16T02:07:55.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-16T02:07:55.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-16T02:07:55.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.295 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.298 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.299 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.301 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.302 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.303 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-16T02:07:55.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-16T02:07:55.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-16T02:07:55.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-16T02:07:55.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.309 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.310 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T02:07:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.315 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.316 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.317 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.318 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.319 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.320 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.321 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-16T02:07:55.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.323 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.325 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.326 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.327 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.328 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.331 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.332 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.332 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.332 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.332 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.333 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.334 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.335 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.336 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-16T02:07:55.336 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.336 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.336 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.336 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:07:55.336 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.336 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.336 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.337 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-16T02:07:55.338 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.339 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-16T02:07:55.340 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.341 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.342 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.342 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.342 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.342 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.342 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.342 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.342 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.342 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.342 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.343 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.343 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.343 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.343 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.343 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.343 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.343 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.343 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.343 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.344 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.344 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.344 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.344 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.344 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.344 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.344 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.345 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.345 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.345 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.345 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.345 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.345 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.345 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.345 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.345 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.346 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.348 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.349 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.350 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.351 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.352 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T02:07:55.354 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.355 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.356 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.359 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.360 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.361 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-16T02:07:55.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.364 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.365 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.366 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.368 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.369 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-16T02:07:55.370 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.371 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.372 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-16T02:07:55.374 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.375 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.378 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.379 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.380 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.381 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.382 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.383 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.384 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.385 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.387 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.388 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.389 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.390 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.391 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.392 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.393 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.394 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.395 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.395 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.395 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.395 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.395 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.395 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.396 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.397 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.397 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.397 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.397 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.397 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.397 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.400 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.400 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.400 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.401 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.403 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.404 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.405 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.406 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.407 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.408 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.410 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.411 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.412 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.413 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.416 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.417 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.418 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.419 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:55.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.422 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.423 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.424 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.425 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.425 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.425 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.425 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.425 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T02:07:55.425 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.425 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.425 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.425 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.427 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.428 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.429 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.430 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.431 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-16T02:07:55.432 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.433 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.434 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-16T02:07:55.434 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.434 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.434 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.434 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.435 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.436 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.437 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.438 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-16T02:07:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-16T02:07:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-16T02:07:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-16T02:07:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-16T02:07:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-16T02:07:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-16T02:07:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.490 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.491 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.492 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-16T02:07:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-16T02:07:55.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.496 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.497 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.498 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.499 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.500 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.501 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.502 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.503 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.504 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-16T02:07:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T02:07:55.506 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.507 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.508 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.509 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.510 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.511 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.512 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.513 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.514 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.515 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.516 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.517 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.518 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.519 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.520 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.521 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.522 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.523 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.524 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.525 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:55.526 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.527 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.528 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.529 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.530 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.532 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.533 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.534 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.535 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.536 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.537 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.538 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-16T02:07:55.539 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-16T02:07:55.539 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.539 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.539 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.539 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.539 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.539 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.539 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.539 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.540 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.541 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.542 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:55.543 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:55.544 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.544 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.544 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.544 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-16T02:07:55.544 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-16T02:07:55.544 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.544 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.544 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.544 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.544 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.545 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.546 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.548 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.549 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.550 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.551 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.553 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-16T02:07:55.554 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-16T02:07:55.555 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.556 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.557 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.558 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.559 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.560 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.561 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.562 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.563 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.565 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.566 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.567 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.568 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-16T02:07:55.569 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.570 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.571 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-16T02:07:55.572 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T02:07:55.572 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.572 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.572 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.572 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.572 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.573 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.574 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.576 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.577 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.578 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.580 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.581 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.582 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.582 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.582 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.582 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.582 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.582 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.582 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.582 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.582 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-16T02:07:55.582 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.584 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.585 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24119, 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "y", 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540701547738038271, 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.587 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.588 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.589 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.591 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.592 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-16T02:07:55.593 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-16T02:07:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-16T02:07:55.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.598 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-16T02:07:55.599 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.600 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T02:07:55.601 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.601 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.601 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.601 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-16T02:07:55.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-16T02:07:55.601 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.601 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.601 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.601 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.604 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.604 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.604 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.604 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-16T02:07:55.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.606 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.607 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.608 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.609 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T02:07:55.610 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-16T02:07:55.611 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T02:07:55.612 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T02:07:55.613 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.617 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.618 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.619 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-16T02:07:55.620 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:55.624 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.625 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.626 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.628 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T02:07:55.629 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T02:07:55.630 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T02:07:55.631 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.632 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.633 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.634 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.635 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-16T02:07:55.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.637 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.639 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.640 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.641 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-16T02:07:55.642 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-16T02:07:55.643 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.644 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.645 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.646 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.647 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T02:07:55.648 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T02:07:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T02:07:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T02:07:55.651 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T02:07:55.652 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.653 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.654 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.655 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T02:07:55.656 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-16T02:07:55.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.658 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.659 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.661 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.662 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.663 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.664 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.665 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.666 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.667 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.668 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.669 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.670 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.671 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.673 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.674 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-16T02:07:55.675 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.678 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.679 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.680 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.681 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.682 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.683 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.684 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-16T02:07:55.685 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-16T02:07:55.686 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-16T02:07:55.687 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.689 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.690 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.691 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.692 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T02:07:55.693 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.694 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.695 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.696 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.697 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.698 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.699 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.700 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.701 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.702 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.703 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.704 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.705 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.706 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.707 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.708 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.709 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.710 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.711 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.712 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.713 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T02:07:55.714 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T02:07:55.715 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T02:07:55.716 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T02:07:55.717 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T02:07:55.718 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-16T02:07:55.719 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.720 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.722 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.723 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.724 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.725 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.726 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.727 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.728 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.729 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.730 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T02:07:55.731 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T02:07:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T02:07:55.733 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T02:07:55.734 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T02:07:55.735 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.736 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.737 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.738 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.739 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.740 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.741 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-16T02:07:55.742 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.743 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.744 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T02:07:55.745 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T02:07:55.746 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.747 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.748 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.751 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.752 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.753 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.754 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.755 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.756 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.757 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.758 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.759 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.760 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.761 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.762 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.763 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.764 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-16T02:07:55.765 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.767 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.768 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.769 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.770 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.771 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.772 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.772 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.772 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.772 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.772 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.772 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.772 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.772 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.773 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.774 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.775 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.776 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.777 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.778 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-16T02:07:55.779 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.780 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.781 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.782 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.783 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.784 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.785 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.786 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.787 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.788 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.789 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.790 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.791 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.792 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-16T02:07:55.793 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-16T02:07:55.796 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T02:07:55.797 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.798 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.799 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.800 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.801 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.802 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.803 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-16T02:07:55.804 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-16T02:07:55.805 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.806 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.806 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.806 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:07:55.806 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.808 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.808 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.809 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-16T02:07:55.810 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:07:55.811 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.812 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.813 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.814 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.815 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.817 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.818 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.819 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.820 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.821 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.822 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.823 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-16T02:07:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.827 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.828 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.829 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.832 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.833 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.834 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.835 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.836 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.837 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.838 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-16T02:07:55.839 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.840 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.840 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.840 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-16T02:07:55.840 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.840 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.840 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.840 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-16T02:07:55.840 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.841 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.842 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.843 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.844 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.845 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.846 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.846 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.846 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.846 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.846 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.846 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.846 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.846 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.847 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.848 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.849 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.850 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.851 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.852 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.855 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.856 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.857 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.858 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.859 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.860 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.861 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.863 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.864 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.865 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.866 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.867 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.868 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.869 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.870 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.871 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.874 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.875 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.877 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.879 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.880 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.881 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.882 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.883 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.884 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.885 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.886 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.887 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.888 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.889 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.891 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.892 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.893 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.896 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.897 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.898 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.899 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.900 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.901 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.902 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.903 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.904 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.905 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.906 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.907 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-16T02:07:55.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.909 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.910 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.911 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.912 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.913 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-16T02:07:55.914 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.915 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-16T02:07:55.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.917 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.918 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.920 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.921 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.923 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.926 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.927 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.928 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.929 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.930 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.931 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.932 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.934 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.935 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.936 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.937 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.938 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.939 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.940 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.941 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.942 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.944 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.945 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.946 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.947 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.948 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.949 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.950 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.951 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.952 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.953 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.954 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.956 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.957 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.958 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.960 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.961 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.962 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.963 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.964 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.965 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.965 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.965 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.965 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.965 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.965 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.965 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-16T02:07:55.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-16T02:07:55.965 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-16T02:07:55.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.967 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.969 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.970 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.971 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.972 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.973 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.974 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.975 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.976 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.977 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.978 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.980 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.981 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.982 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.984 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.985 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.986 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.987 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.988 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.989 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.990 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.991 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:55.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:55.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:55.995 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:55.996 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-16T02:07:55.997 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:55.998 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T02:07:55.999 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.000 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.001 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.002 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.003 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-16T02:07:56.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.005 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.006 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.007 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.009 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-16T02:07:56.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-16T02:07:56.009 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.009 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.009 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.012 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-16T02:07:56.012 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.012 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.012 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.013 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.014 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.015 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.018 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.019 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.020 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.021 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.023 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.024 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.025 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.026 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.027 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.029 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.030 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.033 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.034 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.035 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-16T02:07:56.036 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-16T02:07:56.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-16T02:07:56.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.039 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.040 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.041 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:56.042 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.043 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.044 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.045 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.046 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.047 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-16T02:07:56.048 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-16T02:07:56.049 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-16T02:07:56.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.051 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: "modules": [ 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard", 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: "iostat", 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: "nfs", 2024-09-16T02:07:56.052 INFO:teuthology.orchestra.run.smithi103.stdout: "restful" 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.053 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.054 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.055 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.056 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.057 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-16T02:07:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-16T02:07:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-16T02:07:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T02:07:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-16T02:07:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-16T02:07:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-16T02:07:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-16T02:07:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T02:07:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-16T02:07:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-16T02:07:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-16T02:07:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-16T02:07:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-16T02:07:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.102 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.103 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.104 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-16T02:07:56.105 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.106 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-16T02:07:56.107 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.108 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.109 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.110 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.111 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.112 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.113 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.117 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.118 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.119 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-16T02:07:56.120 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-16T02:07:56.121 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.122 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.123 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-16T02:07:56.124 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-16T02:07:56.125 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.126 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.127 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.128 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.129 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.130 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.131 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.132 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.134 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.135 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.137 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T02:07:56.138 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-16T02:07:56.139 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.140 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.142 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.143 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.144 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.145 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-16T02:07:56.146 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-16T02:07:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-16T02:07:56.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-16T02:07:56.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.150 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.151 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.152 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.153 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.155 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.156 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.157 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.158 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.159 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.160 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.161 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.164 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.165 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.166 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.167 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.168 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.169 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.170 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.171 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.173 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.175 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.177 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T02:07:56.178 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T02:07:56.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-16T02:07:56.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T02:07:56.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.182 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.186 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.187 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.188 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.189 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.190 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.191 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.192 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.193 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.194 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.195 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.196 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T02:07:56.197 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.199 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T02:07:56.200 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.201 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.203 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.204 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.205 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.207 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.209 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.210 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.211 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.212 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.214 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.215 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.216 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.217 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.218 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.219 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.220 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.221 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.222 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-16T02:07:56.223 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.224 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.225 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.226 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.229 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.230 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.231 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.232 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.233 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.234 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.236 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.241 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T02:07:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T02:07:56.243 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.244 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.245 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.246 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.247 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.247 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.247 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.247 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.247 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.247 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.247 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.247 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.248 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.250 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.251 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-16T02:07:56.252 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.255 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.256 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-16T02:07:56.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.258 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.260 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.262 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.263 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.264 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-16T02:07:56.265 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-16T02:07:56.266 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-16T02:07:56.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.271 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.272 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.273 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.274 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.274 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.274 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.274 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.274 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.274 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.274 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.274 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.274 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.274 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.276 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.277 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.278 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.279 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.279 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.279 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.279 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.279 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.279 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.279 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.279 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.280 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.281 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T02:07:56.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.284 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.286 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.288 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.289 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.289 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.289 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.289 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.289 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.289 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.289 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.289 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.290 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.291 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.292 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.293 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.295 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.297 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.298 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.299 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.300 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-16T02:07:56.301 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.302 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.303 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.304 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.305 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.308 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.309 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.311 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.312 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.314 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.315 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.316 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.317 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.318 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.319 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.320 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.321 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.322 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.323 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.324 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.325 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-16T02:07:56.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-16T02:07:56.328 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.329 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.330 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.331 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.332 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.333 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.334 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.335 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.336 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.337 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.338 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.339 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.340 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.341 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.342 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.343 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.344 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.346 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-16T02:07:56.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.348 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.349 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.350 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.351 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.352 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.354 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-16T02:07:56.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.357 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.358 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.359 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.360 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.361 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.362 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.363 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.364 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-16T02:07:56.366 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T02:07:56.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.368 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.369 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.371 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.372 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-16T02:07:56.373 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-16T02:07:56.374 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.375 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.376 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.377 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.378 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.379 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.380 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.381 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.382 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.383 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.384 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.386 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.387 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.388 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.389 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.390 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.391 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.392 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.393 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.394 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.395 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.396 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.397 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.399 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T02:07:56.400 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.401 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.402 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.404 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.407 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.408 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.409 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.413 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.414 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.415 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.418 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.419 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.420 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.421 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.422 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-16T02:07:56.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-16T02:07:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-16T02:07:56.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.426 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.427 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.428 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.429 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.430 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.431 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.432 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.433 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.435 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.436 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.437 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.438 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.439 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.439 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.439 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.439 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.439 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.439 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.439 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.439 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.439 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.440 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.441 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.442 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.443 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.448 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.449 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.450 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.451 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.452 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.455 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.456 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.457 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.458 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.459 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.460 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.461 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-16T02:07:56.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-16T02:07:56.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.466 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.467 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.467 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.467 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-16T02:07:56.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-16T02:07:56.467 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.467 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.467 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.467 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.467 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.468 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.469 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.470 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.471 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.472 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.473 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-16T02:07:56.474 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-16T02:07:56.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.476 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.477 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.478 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.480 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.481 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.483 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.484 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.485 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.486 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.487 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.488 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.492 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.493 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.494 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.494 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.494 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.494 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-16T02:07:56.494 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-16T02:07:56.494 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-16T02:07:56.494 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-16T02:07:56.494 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.496 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.497 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.498 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.499 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-16T02:07:56.501 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-16T02:07:56.502 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-16T02:07:56.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.505 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.506 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "services": { 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard": "https://172.21.15.103:8443/" 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "always_on_modules": { 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "octopus": [ 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-16T02:07:56.507 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "pacific": [ 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-16T02:07:56.508 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "quincy": [ 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-16T02:07:56.509 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "reef": [ 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-16T02:07:56.510 INFO:teuthology.orchestra.run.smithi103.stdout: "squid": [ 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.511 INFO:teuthology.orchestra.run.smithi103.stdout: "last_failure_osd_epoch": 3, 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: "active_clients": [ 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 333054008 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "libcephsqlite", 2024-09-16T02:07:56.512 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 945987873 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-16T02:07:56.513 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2356855531 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 846360008 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-16T02:07:56.514 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:56.515 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-16T02:07:56.515 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:07:56.516 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:55 smithi172 ceph-mon[35340]: pgmap v414: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-16T02:07:56.516 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:55 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/690723923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:07:56.516 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:55 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/249429857' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T02:07:56.516 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:55 smithi136 ceph-mon[35323]: pgmap v414: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-16T02:07:56.516 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:55 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/690723923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:07:56.516 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:55 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/249429857' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T02:07:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.102+0000 7f477d600640 1 -- 172.21.15.103:0/249429857 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f475c0a32d0 msgr2=0x7f475c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.102+0000 7f477d600640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f475c0a32d0 0x7f475c0a5790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4768000c00 tx=0x7f47680095f0 comp rx=0 tx=0).stop 2024-09-16T02:07:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.102+0000 7f477d600640 1 -- 172.21.15.103:0/249429857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47780761d0 msgr2=0x7f477819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:07:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.102+0000 7f477d600640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47780761d0 0x7f477819a5e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f476000e9f0 tx=0x7f476000edf0 comp rx=0 tx=0).stop 2024-09-16T02:07:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.103+0000 7f477d600640 1 -- 172.21.15.103:0/249429857 shutdown_connections 2024-09-16T02:07:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.103+0000 7f477d600640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f475c0a32d0 0x7f475c0a5790 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:56.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.103+0000 7f477d600640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47781099c0 0x7f47781a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:56.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.103+0000 7f477d600640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47780761d0 0x7f477819a5e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:56.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.103+0000 7f477d600640 1 --2- 172.21.15.103:0/249429857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4778074bf0 0x7f477819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:07:56.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.103+0000 7f477d600640 1 -- 172.21.15.103:0/249429857 >> 172.21.15.103:0/249429857 conn(0x7f47780fd2b0 msgr2=0x7f4778105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:07:56.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.103+0000 7f477d600640 1 -- 172.21.15.103:0/249429857 shutdown_connections 2024-09-16T02:07:56.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:07:55.103+0000 7f477d600640 1 -- 172.21.15.103:0/249429857 wait complete. 2024-09-16T02:07:56.581 DEBUG:tasks.check_counter:Getting stats from a 2024-09-16T02:07:56.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-16T02:07:56.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:55 smithi103 ceph-mon[30398]: pgmap v414: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-16T02:07:56.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:55 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/690723923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T02:07:56.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:55 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/249429857' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T02:07:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:57.499 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:07:57.499 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:07:57.499 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 82, 2024-09-16T02:07:57.499 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 77, 2024-09-16T02:07:57.499 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 716976, 2024-09-16T02:07:57.499 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 46272, 2024-09-16T02:07:57.499 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.023706646, 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.007220719, 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.021456710, 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 76, 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.005801904, 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000076340 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:07:57.500 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 716976, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 46272 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 5, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 60, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3232, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 89248, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.010075180, 2024-09-16T02:07:57.501 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.004850460, 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.004503094, 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 59, 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.009608570, 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000162857 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-16T02:07:57.502 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 89248 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.000795664, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000632588, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:07:57.503 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-16T02:07:57.504 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:07:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:07:57.506 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:07:57.507 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148823, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 27, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:07:57.508 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 83, 2024-09-16T02:07:57.509 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 705434, 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 83, 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 83, 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 705434, 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:57.510 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:07:58.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:57 smithi103 ceph-mon[30398]: pgmap v415: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:07:58.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:57 smithi136 ceph-mon[35323]: pgmap v415: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:07:58.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:57 smithi172 ceph-mon[35340]: pgmap v415: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:07:58.300 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-16T02:07:58.301 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-16T02:07:58.301 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-16T02:07:58.301 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-16T02:07:58.301 DEBUG:tasks.check_counter:Getting stats from d 2024-09-16T02:07:58.301 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-16T02:07:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:07:59.310 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:07:59.310 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:07:59.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-09-16T02:07:59.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-09-16T02:07:59.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-09-16T02:07:59.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-09-16T02:07:59.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-16T02:07:59.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-16T02:07:59.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.003793124, 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000901425, 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:07:59.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 91, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 84, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 721408, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 49920, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.022264204, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.005709053, 2024-09-16T02:07:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.023349488, 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 84, 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.012071838, 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000143712 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 721408, 2024-09-16T02:07:59.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 49920 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 63, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 160, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 89680, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.007432302, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.005299607, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001319200, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:07:59.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 61, 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.003302103, 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000054132 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 89680 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-16T02:07:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:07:59.316 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:07:59.317 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 149854, 2024-09-16T02:07:59.318 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 28, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-16T02:07:59.319 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 88, 2024-09-16T02:07:59.320 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 706084, 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 88, 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 88, 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 706084, 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:07:59.321 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:08:00.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:07:59 smithi103 ceph-mon[30398]: pgmap v416: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:00.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:07:59 smithi136 ceph-mon[35323]: pgmap v416: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:00.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:07:59 smithi172 ceph-mon[35340]: pgmap v416: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:00.066 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-16T02:08:00.066 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-16T02:08:00.067 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-16T02:08:00.067 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-16T02:08:00.067 DEBUG:tasks.check_counter:Getting stats from g 2024-09-16T02:08:00.067 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-16T02:08:00.437 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:08:01.021 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:08:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.003307373, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000543316, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:08:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:08:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:08:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:08:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:08:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:08:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:08:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:08:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:08:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 84, 2024-09-16T02:08:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 79, 2024-09-16T02:08:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 717520, 2024-09-16T02:08:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 47552, 2024-09-16T02:08:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-16T02:08:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-16T02:08:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.021127487, 2024-09-16T02:08:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.007179220, 2024-09-16T02:08:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.022135400, 2024-09-16T02:08:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 78, 2024-09-16T02:08:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.005718260, 2024-09-16T02:08:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000073311 2024-09-16T02:08:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:08:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:08:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:08:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 717520, 2024-09-16T02:08:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 47552 2024-09-16T02:08:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:08:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 5, 2024-09-16T02:08:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 62, 2024-09-16T02:08:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3232, 2024-09-16T02:08:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 89600, 2024-09-16T02:08:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-16T02:08:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-16T02:08:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.009940121, 2024-09-16T02:08:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.004745347, 2024-09-16T02:08:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.004436411, 2024-09-16T02:08:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 61, 2024-09-16T02:08:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.009309219, 2024-09-16T02:08:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000152610 2024-09-16T02:08:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:08:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:08:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:08:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-16T02:08:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 89600 2024-09-16T02:08:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-16T02:08:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-16T02:08:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-16T02:08:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-16T02:08:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:08:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-16T02:08:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:08:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:08:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:08:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:08:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:08:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:08:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:08:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:08:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:08:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:08:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:08:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:08:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:08:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-16T02:08:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:08:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-16T02:08:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:08:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:08:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:08:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:08:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:08:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:08:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:08:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-16T02:08:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:08:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:08:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:08:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-16T02:08:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-16T02:08:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-16T02:08:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:08:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:08:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:08:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:08:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148823, 2024-09-16T02:08:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 27, 2024-09-16T02:08:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:08:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-16T02:08:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-16T02:08:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-16T02:08:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:08:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-16T02:08:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:08:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-16T02:08:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-16T02:08:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-16T02:08:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:08:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:08:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-16T02:08:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-16T02:08:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-16T02:08:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-16T02:08:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-16T02:08:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-16T02:08:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-16T02:08:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-16T02:08:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:08:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-16T02:08:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-16T02:08:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-16T02:08:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 85, 2024-09-16T02:08:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 705694, 2024-09-16T02:08:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 85, 2024-09-16T02:08:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-16T02:08:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-16T02:08:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 85, 2024-09-16T02:08:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 705694, 2024-09-16T02:08:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-16T02:08:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:08:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:08:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:08:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:08:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:08:01.041 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:08:01.824 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-16T02:08:01.824 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-16T02:08:01.824 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-16T02:08:01.824 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-16T02:08:01.825 DEBUG:tasks.check_counter:Getting stats from j 2024-09-16T02:08:01.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-16T02:08:02.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:01 smithi103 ceph-mon[30398]: pgmap v417: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:08:02.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:01 smithi136 ceph-mon[35323]: pgmap v417: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:08:02.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:01 smithi172 ceph-mon[35340]: pgmap v417: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:08:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:08:02.795 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-16T02:08:02.795 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:08:02.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-09-16T02:08:02.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-09-16T02:08:02.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.004637877, 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000422990, 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:08:02.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1, 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 64, 2024-09-16T02:08:02.797 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 160, 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 89856, 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.007590460, 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.005019229, 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001577305, 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 62, 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.004117088, 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000066404 2024-09-16T02:08:02.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 89856 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 93, 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 86, 2024-09-16T02:08:02.799 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 721952, 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 51200, 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.022961349, 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.007474319, 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.022124511, 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 86, 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.011751957, 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000136650 2024-09-16T02:08:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 721952, 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 51200 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.801 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:08:02.802 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-16T02:08:02.803 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148841, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 28, 2024-09-16T02:08:02.804 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-16T02:08:02.805 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 90, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 706344, 2024-09-16T02:08:02.806 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 90, 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 90, 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 706344, 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-16T02:08:02.807 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-16T02:08:03.489 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-16T02:08:03.490 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-16T02:08:03.490 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-16T02:08:03.490 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-16T02:08:03.490 DEBUG:tasks.check_counter:Getting stats from b 2024-09-16T02:08:03.490 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-16T02:08:03.748 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:08:03.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:03 smithi103 ceph-mon[30398]: pgmap v418: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T02:08:03.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:03 smithi136 ceph-mon[35323]: pgmap v418: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T02:08:04.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:03 smithi172 ceph-mon[35340]: pgmap v418: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T02:08:04.240 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T02:08:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:08:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2377, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2361, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1389445, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 5162544, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 26, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 6, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.190083265, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.064497861, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.077328493, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.045862374, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2361, 2024-09-16T02:08:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.138896638, 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000058829 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 684, 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000147462, 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000215 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 806144, 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 59888 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:08:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 7733, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 12396, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 2556950, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 9417332, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 32, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.459110278, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.186562187, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.265891681, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.071857932, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 12394, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.357786463, 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000028867 2024-09-16T02:08:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 7155, 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.002642793, 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000369 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 61104, 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 214800 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2661, 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2745, 2024-09-16T02:08:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 678839, 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 5339788, 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 12, 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 6, 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.212123775, 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.090104841, 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.057020389, 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.055212276, 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2743, 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.224998677, 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000082026 2024-09-16T02:08:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 571, 2024-09-16T02:08:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000111128, 2024-09-16T02:08:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000194 2024-09-16T02:08:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:08:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:08:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-16T02:08:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4583, 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2.056499526, 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000448723 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-16T02:08:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-16T02:08:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 5418, 2024-09-16T02:08:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.191740238, 2024-09-16T02:08:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000035389 2024-09-16T02:08:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-16T02:08:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 2470, 2024-09-16T02:08:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 2470, 2024-09-16T02:08:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-16T02:08:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2470, 2024-09-16T02:08:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2.482703811, 2024-09-16T02:08:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001005143 2024-09-16T02:08:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-16T02:08:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-09-16T02:08:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-09-16T02:08:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-16T02:08:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 48, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 70, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 6, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 64, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 61, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 3, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 3, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 2, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 2495, 2024-09-16T02:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 2479, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-16T02:08:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 38, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 38, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 25, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 3, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 22, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 13, 2024-09-16T02:08:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 24, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 3639, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 2442, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 1212, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 13, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 1240 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 11, 2024-09-16T02:08:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 20, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 1, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 1212, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 1211, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:08:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 3, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 6127, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 2489, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 2489, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 3638, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 2412, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 16, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 10, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 10, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 6, 2024-09-16T02:08:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 1, 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 3, 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 9407021, 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 16816988, 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2450, 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 3.893449694, 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001589163 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 24, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 1238, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 1214, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 19, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 24, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 5, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 70, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 1243, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 1173, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 3, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 1227, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 1224, 2024-09-16T02:08:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 56884, 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 231712 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 2531, 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 1, 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 2470, 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 14, 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 2, 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1210, 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.993272766, 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000820886 2024-09-16T02:08:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 16, 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.003662573, 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000228910 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000289497, 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000144748 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-16T02:08:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-16T02:08:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 12, 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.021048276, 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001754023 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000197450, 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000197450 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6, 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.008321384, 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001386897 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.003483401, 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001741700 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6, 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.031107188, 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.005184531 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000917902, 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000917902 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 8, 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.020500963, 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002562620 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-16T02:08:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1206, 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.399902411, 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001160781 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-16T02:08:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 3, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 2, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 2182, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 2182, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 149, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-16T02:08:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-16T02:08:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:08:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 308305, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 134, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 616, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 7, 2024-09-16T02:08:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7984, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 165, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 134290, 2024-09-16T02:08:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 1173, 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 6801, 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 12946020, 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 37, 2024-09-16T02:08:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 6801, 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6801, 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 32.712259556, 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.004809919 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6801, 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 7091 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 6801, 2024-09-16T02:08:04.280 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 74, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 6727, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 28, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 81, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 33, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 2480, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 49, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 4013, 2024-09-16T02:08:04.281 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 41, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 162, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-16T02:08:04.282 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:08:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-09-16T02:08:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-16T02:08:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-16T02:08:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-16T02:08:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-16T02:08:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-16T02:08:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-16T02:08:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-16T02:08:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-16T02:08:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 81, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 4, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 12, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-09-16T02:08:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 176, 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 0, 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 28 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 12, 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 7, 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 3497, 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 1211, 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:08:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:04.287 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-16T02:08:04.287 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:04.287 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 12767, 2024-09-16T02:08:04.287 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 3535891, 2024-09-16T02:08:04.287 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:04.287 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 12767, 2024-09-16T02:08:04.287 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-16T02:08:04.287 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-16T02:08:04.287 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 12767, 2024-09-16T02:08:04.287 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 3535891, 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-16T02:08:04.288 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 6801, 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 13068331, 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6801, 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 13068331, 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.289 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 6801, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 6801, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6801, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 6801, 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 1211, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 122311, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 1211, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-16T02:08:04.291 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 30, 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 122311, 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x556ed307a960": { 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:04.292 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 6127, 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 12622684, 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 6127, 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2450, 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 12622684, 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:04.293 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T02:08:05.019 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-16T02:08:05.020 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-16T02:08:05.020 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=11 2024-09-16T02:08:05.020 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-16T02:08:05.020 DEBUG:tasks.check_counter:Getting stats from e 2024-09-16T02:08:05.020 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-16T02:08:05.272 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:08:05.765 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T02:08:05.765 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:08:05.765 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 267, 2024-09-16T02:08:05.765 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 269, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 72464, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 84970, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.025225989, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.009524199, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.006310778, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.006125276, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 267, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.012728004, 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000047670 2024-09-16T02:08:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 164, 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 224, 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 42368, 2024-09-16T02:08:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 232391, 2024-09-16T02:08:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 5, 2024-09-16T02:08:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-09-16T02:08:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.019847220, 2024-09-16T02:08:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.007141785, 2024-09-16T02:08:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.005892523, 2024-09-16T02:08:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.004813839, 2024-09-16T02:08:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 223, 2024-09-16T02:08:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.015005928, 2024-09-16T02:08:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000067291 2024-09-16T02:08:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 181824 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 357, 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 350, 2024-09-16T02:08:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 829088, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 133692, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 3, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.051471927, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.016174946, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.029562705, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.010217186, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 349, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.021021471, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000060233 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 742400, 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 51648 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-16T02:08:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-16T02:08:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-16T02:08:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1184, 2024-09-16T02:08:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.033737982, 2024-09-16T02:08:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000028494 2024-09-16T02:08:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 0, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 0, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 0, 2024-09-16T02:08:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-09-16T02:08:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 0, 2024-09-16T02:08:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-09-16T02:08:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-09-16T02:08:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-09-16T02:08:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 1, 2024-09-16T02:08:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-09-16T02:08:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-09-16T02:08:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-09-16T02:08:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-09-16T02:08:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:08:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-16T02:08:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-09-16T02:08:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:08:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:08:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-09-16T02:08:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:08:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-09-16T02:08:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-09-16T02:08:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-16T02:08:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-09-16T02:08:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-09-16T02:08:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-09-16T02:08:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-09-16T02:08:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-16T02:08:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-09-16T02:08:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 7, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 2, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 4196271, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 4196130, 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-16T02:08:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 7 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 3, 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 3, 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 0, 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 1, 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 1, 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 0, 2024-09-16T02:08:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 0, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 0, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 0, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 47744, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215328 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-09-16T02:08:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-16T02:08:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-16T02:08:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-16T02:08:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-16T02:08:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.357 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:05 smithi172 ceph-mon[35340]: pgmap v419: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:06.357 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:05 smithi103 ceph-mon[30398]: pgmap v419: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-16T02:08:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-16T02:08:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-16T02:08:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-16T02:08:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-16T02:08:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-16T02:08:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:08:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 3, 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 2, 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 2, 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-16T02:08:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 2853, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:08:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:08:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:08:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 189217, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 48, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7984, 2024-09-16T02:08:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 165, 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 8512, 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 13, 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-16T02:08:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-16T02:08:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-16T02:08:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-16T02:08:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 684, 2024-09-16T02:08:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 21, 2024-09-16T02:08:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 3, 2024-09-16T02:08:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 684, 2024-09-16T02:08:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-16T02:08:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 684, 2024-09-16T02:08:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.353991986, 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000517532 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 684, 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 685 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 684, 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 684, 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-16T02:08:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 415, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 267, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-16T02:08:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-16T02:08:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-16T02:08:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 1, 2024-09-16T02:08:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-09-16T02:08:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:08:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-16T02:08:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:08:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-16T02:08:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-16T02:08:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-16T02:08:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:08:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-16T02:08:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:08:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-09-16T02:08:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-16T02:08:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-16T02:08:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-16T02:08:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-16T02:08:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-16T02:08:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-16T02:08:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 81, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 4, 2024-09-16T02:08:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 5, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 5, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:08:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 784, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 865237, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 784, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 784, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 865237, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 684, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 1972, 2024-09-16T02:08:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 684, 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1972, 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-16T02:08:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 684, 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 684, 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 684, 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 684, 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:08:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x55fe3ea96b40": { 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-16T02:08:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-16T02:08:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:06.386 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T02:08:06.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:05 smithi136 ceph-mon[35323]: pgmap v419: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:06.578 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-16T02:08:06.578 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-16T02:08:06.578 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-16T02:08:06.578 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-16T02:08:06.578 DEBUG:tasks.check_counter:Getting stats from h 2024-09-16T02:08:06.579 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-16T02:08:06.824 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 34, 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 36, 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 10130, 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 84485, 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 15, 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 7, 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.010401394, 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.001639691, 2024-09-16T02:08:07.341 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.006452124, 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000542451, 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 33, 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.035121199, 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001064278 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 13, 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000001706, 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000131 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 4000, 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 1184 2024-09-16T02:08:07.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 346, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 325, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 110054, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 253817, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 7, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.037288620, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.018811391, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.017825438, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.001716920, 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:07.343 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 325, 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.027344107, 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000084135 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 578, 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000109507, 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000189 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 182816 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 141, 2024-09-16T02:08:07.344 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 130, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 758527, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 114265, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 9, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.021838398, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.006678589, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.023330791, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000231658, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 129, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.019848030, 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000153860 2024-09-16T02:08:07.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4, 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000262, 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000065 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 755728, 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 55808 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-16T02:08:07.346 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 48, 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.001767028, 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000036813 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000013374, 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000006687 2024-09-16T02:08:07.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-09-16T02:08:07.348 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 11, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 17, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 1, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 16, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 13, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 8, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 3, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 9, 2024-09-16T02:08:07.349 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 6, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 3, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-16T02:08:07.350 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:08:07.351 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 15, 2024-09-16T02:08:07.352 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 9, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 10, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-16T02:08:07.353 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 4, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-09-16T02:08:07.354 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 4, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 2, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 2, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 4195426, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-09-16T02:08:07.355 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-16T02:08:07.356 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3, 2024-09-16T02:08:07.356 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.006815810, 2024-09-16T02:08:07.356 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002271936 2024-09-16T02:08:07.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.356 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-09-16T02:08:07.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.356 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-16T02:08:07.356 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 21, 2024-09-16T02:08:07.356 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 27, 2024-09-16T02:08:07.356 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 6, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 20, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 26, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 6, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 17, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 25, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 8, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 49928, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 223520 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.357 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 4, 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.358 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.359 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.360 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.362 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-16T02:08:07.362 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.362 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.362 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.362 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-16T02:08:07.362 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.363 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.364 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.366 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.367 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:08:07.369 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 0, 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 1, 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 1, 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 0, 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-09-16T02:08:07.370 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 96, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:08:07.371 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:08:07.372 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 230119, 2024-09-16T02:08:07.373 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 73, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7984, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 165, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-16T02:08:07.374 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 85085, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 331, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 38, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 11334, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 29, 2024-09-16T02:08:07.375 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 38, 2024-09-16T02:08:07.376 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-16T02:08:07.376 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 38, 2024-09-16T02:08:07.376 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.219995018, 2024-09-16T02:08:07.376 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.005789342 2024-09-16T02:08:07.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.376 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-16T02:08:07.376 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-16T02:08:07.376 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 38, 2024-09-16T02:08:07.376 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 60 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 38, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 0, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 38, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 0, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 10, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 0, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 3, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 6, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-16T02:08:07.377 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 6, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 20, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-16T02:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-16T02:08:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 81, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 4, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 9, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 9, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-16T02:08:07.380 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 14, 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 0, 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 1 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-16T02:08:07.381 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 517, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 822506, 2024-09-16T02:08:07.382 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 517, 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 517, 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 822506, 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:07.383 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 38, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 11334, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 38, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 11334, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.384 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 38, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 38, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 38, 2024-09-16T02:08:07.385 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 38, 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:07.386 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x5586261fa960": { 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:07.387 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 4, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1122, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 4, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 3, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1122, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:07.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:07.389 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:07.389 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T02:08:07.960 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:07 smithi136 ceph-mon[35323]: pgmap v420: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:08:07.986 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-16T02:08:07.986 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-16T02:08:07.986 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-16T02:08:07.986 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=15 2024-09-16T02:08:07.986 DEBUG:tasks.check_counter:Getting stats from k 2024-09-16T02:08:07.987 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-16T02:08:08.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:07 smithi103 ceph-mon[30398]: pgmap v420: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:08:08.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:07 smithi172 ceph-mon[35340]: pgmap v420: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:08:08.229 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T02:08:08.754 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T02:08:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:08:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 4, 2024-09-16T02:08:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 4, 2024-09-16T02:08:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 3072, 2024-09-16T02:08:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 448, 2024-09-16T02:08:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 4, 2024-09-16T02:08:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 0, 2024-09-16T02:08:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.002967747, 2024-09-16T02:08:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.000035302, 2024-09-16T02:08:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.002209319, 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3, 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.003939443, 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001313147 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-16T02:08:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 93, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 88, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 720032, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 52832, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 2, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 1, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.019257705, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.006204436, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.020199644, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 87, 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.005321842, 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000061170 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 720032, 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 52832 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1, 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 66, 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 160, 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 90208, 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 3, 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.004478660, 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.002906718, 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.000756410, 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 64, 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.004221609, 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000065962 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 90208 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-16T02:08:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 0, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:08:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:08:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:08:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:08:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-16T02:08:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:08:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:08:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:08:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-16T02:08:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-16T02:08:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-16T02:08:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:08:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 148840, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 28, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:08:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 0, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 0, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 928, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 2, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-16T02:08:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 94, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 706928, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 94, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 94, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 706928, 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-16T02:08:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-16T02:08:08.767 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-16T02:08:08.767 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-16T02:08:08.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:08.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T02:08:08.767 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T02:08:09.452 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-16T02:08:09.452 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-16T02:08:09.452 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-16T02:08:09.452 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-16T02:08:09.452 DEBUG:tasks.check_counter:Getting stats from c 2024-09-16T02:08:09.452 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-16T02:08:09.715 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:08:09.773 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:09 smithi136 ceph-mon[35323]: pgmap v421: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:09.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:09 smithi172 ceph-mon[35340]: pgmap v421: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:10.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:09 smithi103 ceph-mon[30398]: pgmap v421: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:10.173 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-16T02:08:10.174 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:08:10.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 600, 2024-09-16T02:08:10.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 593, 2024-09-16T02:08:10.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 5087310, 2024-09-16T02:08:10.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 210654, 2024-09-16T02:08:10.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 5, 2024-09-16T02:08:10.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 1, 2024-09-16T02:08:10.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.069450014, 2024-09-16T02:08:10.175 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.021633808, 2024-09-16T02:08:10.175 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.033793201, 2024-09-16T02:08:10.175 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.015437719, 2024-09-16T02:08:10.175 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:10.175 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 592, 2024-09-16T02:08:10.175 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.024774278, 2024-09-16T02:08:10.175 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000041848 2024-09-16T02:08:10.175 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.175 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:10.175 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.176 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.176 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.176 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 742896, 2024-09-16T02:08:10.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 51968 2024-09-16T02:08:10.176 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.176 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:08:10.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 81, 2024-09-16T02:08:10.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 141, 2024-09-16T02:08:10.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 4257588, 2024-09-16T02:08:10.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 207674, 2024-09-16T02:08:10.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 5, 2024-09-16T02:08:10.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 4, 2024-09-16T02:08:10.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.014283985, 2024-09-16T02:08:10.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.005081418, 2024-09-16T02:08:10.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.006234198, 2024-09-16T02:08:10.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.001750400, 2024-09-16T02:08:10.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:10.177 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 140, 2024-09-16T02:08:10.177 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.014164662, 2024-09-16T02:08:10.178 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000101176 2024-09-16T02:08:10.178 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.178 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:10.178 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.178 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.178 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.178 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.178 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-16T02:08:10.178 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 183440 2024-09-16T02:08:10.178 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 265, 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 266, 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 4260052, 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 83959, 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 5, 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 4, 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.027893196, 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.009166501, 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.008210881, 2024-09-16T02:08:10.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.008091598, 2024-09-16T02:08:10.180 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:10.180 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 264, 2024-09-16T02:08:10.180 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.012139303, 2024-09-16T02:08:10.180 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000045982 2024-09-16T02:08:10.180 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.180 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:10.180 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.180 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.180 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.180 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-16T02:08:10.181 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1391, 2024-09-16T02:08:10.182 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.042297557, 2024-09-16T02:08:10.182 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000030408 2024-09-16T02:08:10.182 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:10.182 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.182 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-09-16T02:08:10.182 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-16T02:08:10.182 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-16T02:08:10.182 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.182 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.182 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.183 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:10.183 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.183 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-09-16T02:08:10.183 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 0, 2024-09-16T02:08:10.183 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 0, 2024-09-16T02:08:10.183 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-09-16T02:08:10.183 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.184 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.184 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.184 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.184 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-09-16T02:08:10.184 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 0, 2024-09-16T02:08:10.184 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-09-16T02:08:10.184 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-09-16T02:08:10.184 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 0, 2024-09-16T02:08:10.184 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-09-16T02:08:10.184 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 1234, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 321, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 913, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 775, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 2, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 0, 2024-09-16T02:08:10.185 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 0, 2024-09-16T02:08:10.186 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 4, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 234, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:08:10.187 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 0, 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-09-16T02:08:10.188 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-09-16T02:08:10.189 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-09-16T02:08:10.190 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 0, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 6127, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 0, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-09-16T02:08:10.191 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 6, 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 9, 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 9407021, 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 16816988, 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 16813566, 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 6127 2024-09-16T02:08:10.192 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 22, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 1233, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 1211, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 17, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 22, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 5, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 1234, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 1240, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 6, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-09-16T02:08:10.193 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 0, 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 0, 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 50652, 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 215328 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 0, 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 0, 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 0, 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 0, 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-09-16T02:08:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.195 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.196 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.196 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-09-16T02:08:10.196 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.196 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.196 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.196 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.196 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-09-16T02:08:10.196 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.196 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.196 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.197 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.197 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-09-16T02:08:10.197 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.197 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.197 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.197 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.197 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-09-16T02:08:10.197 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.197 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.197 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.198 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.199 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-09-16T02:08:10.199 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.199 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.199 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.199 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.199 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-09-16T02:08:10.199 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.199 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.199 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.199 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.200 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-09-16T02:08:10.201 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.202 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.202 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.202 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.202 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-09-16T02:08:10.202 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.202 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.202 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.202 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.202 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-09-16T02:08:10.202 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.203 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.203 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.203 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.203 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-09-16T02:08:10.203 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.203 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.203 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.203 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.203 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-09-16T02:08:10.203 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-09-16T02:08:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-09-16T02:08:10.204 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.205 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.206 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.206 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.206 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-09-16T02:08:10.206 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.206 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.206 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.206 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.206 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-09-16T02:08:10.206 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.207 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.208 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.208 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.208 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-09-16T02:08:10.208 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.208 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.208 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.208 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.208 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-09-16T02:08:10.208 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.208 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.209 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.210 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.210 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:10.210 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.210 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-09-16T02:08:10.210 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 1, 2024-09-16T02:08:10.210 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 3, 2024-09-16T02:08:10.210 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 2, 2024-09-16T02:08:10.210 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 2, 2024-09-16T02:08:10.210 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-09-16T02:08:10.210 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 0, 2024-09-16T02:08:10.211 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 0, 2024-09-16T02:08:10.211 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 2820, 2024-09-16T02:08:10.211 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-09-16T02:08:10.211 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.211 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-16T02:08:10.211 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:08:10.211 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-16T02:08:10.211 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:08:10.211 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:08:10.211 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:08:10.212 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-16T02:08:10.213 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:08:10.214 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:08:10.214 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:08:10.214 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-16T02:08:10.214 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-16T02:08:10.214 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-16T02:08:10.214 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:08:10.214 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:08:10.214 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:08:10.214 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:08:10.214 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 197390, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 55, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 7984, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 165, 2024-09-16T02:08:10.215 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:08:10.216 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:08:10.216 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-16T02:08:10.216 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-16T02:08:10.216 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 939898, 2024-09-16T02:08:10.216 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 5296, 2024-09-16T02:08:10.216 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-16T02:08:10.216 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-16T02:08:10.216 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-16T02:08:10.216 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-16T02:08:10.216 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 839, 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 26, 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 4, 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 839, 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 839, 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.927976563, 2024-09-16T02:08:10.217 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001106050 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 839, 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 840 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 839, 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 839, 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 0, 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-09-16T02:08:10.218 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 521, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 0, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 315, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 0, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 0, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 0, 2024-09-16T02:08:10.219 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 2, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 0, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-09-16T02:08:10.220 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-09-16T02:08:10.221 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-09-16T02:08:10.222 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-09-16T02:08:10.222 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-09-16T02:08:10.222 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-09-16T02:08:10.222 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-09-16T02:08:10.222 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-09-16T02:08:10.222 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-09-16T02:08:10.222 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-09-16T02:08:10.222 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-09-16T02:08:10.222 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 81, 2024-09-16T02:08:10.222 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 4, 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 6, 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 6, 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 0, 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 2, 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 0 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-09-16T02:08:10.223 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 0, 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:08:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 942, 2024-09-16T02:08:10.225 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 13511949, 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 942, 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 942, 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 13511949, 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.226 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 839, 2024-09-16T02:08:10.227 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 12622694, 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 839, 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 12622694, 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:10.228 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 839, 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 839, 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 839, 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 839, 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:10.229 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:10.230 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:10.231 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x5604fb046b40": { 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-16T02:08:10.232 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-16T02:08:10.233 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:10.233 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:10.233 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:10.233 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:10.233 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:10.233 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:10.233 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-16T02:08:10.966 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-16T02:08:10.967 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-16T02:08:10.967 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-16T02:08:10.967 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-16T02:08:10.967 DEBUG:tasks.check_counter:Getting stats from f 2024-09-16T02:08:10.967 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-16T02:08:11.215 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:08:11.725 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 368, 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 359, 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 834448, 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 136530, 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 5, 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 1, 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.053984045, 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.016006206, 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.030323681, 2024-09-16T02:08:11.726 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.010457073, 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 359, 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.025308450, 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000070497 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 746080, 2024-09-16T02:08:11.727 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 52896 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 3, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 66, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 716, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 185482, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 3, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.004348609, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.002529107, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.001052084, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000078199, 2024-09-16T02:08:11.728 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 64, 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.004391594, 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000068618 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 184784 2024-09-16T02:08:11.729 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.730 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:08:11.733 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 433, 2024-09-16T02:08:11.734 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 434, 2024-09-16T02:08:11.734 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 112673, 2024-09-16T02:08:11.734 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 137383, 2024-09-16T02:08:11.734 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-09-16T02:08:11.734 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 3, 2024-09-16T02:08:11.734 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.033859495, 2024-09-16T02:08:11.734 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.011672705, 2024-09-16T02:08:11.734 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.009090464, 2024-09-16T02:08:11.734 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.010536044, 2024-09-16T02:08:11.734 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 432, 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.018026680, 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000041728 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T02:08:11.735 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1224, 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.036328257, 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000029679 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.736 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 0, 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 0, 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.737 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 0, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 0, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 0, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 0, 2024-09-16T02:08:11.738 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 1, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-09-16T02:08:11.739 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 0, 2024-09-16T02:08:11.740 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:08:11.741 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 0, 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 0, 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 0, 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 0, 2024-09-16T02:08:11.742 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-09-16T02:08:11.743 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-09-16T02:08:11.744 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 0, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 4, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 0, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-09-16T02:08:11.745 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-09-16T02:08:11.746 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 4195426, 2024-09-16T02:08:11.746 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 4195285, 2024-09-16T02:08:11.746 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-09-16T02:08:11.746 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.746 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.746 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.747 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.747 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 4 2024-09-16T02:08:11.747 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.747 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-09-16T02:08:11.747 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 3, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 3, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 0, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 1, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 1, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 0, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 0, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 0, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 0, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 0, 2024-09-16T02:08:11.748 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 0, 2024-09-16T02:08:11.749 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 48164, 2024-09-16T02:08:11.749 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 215328 2024-09-16T02:08:11.749 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.749 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-09-16T02:08:11.749 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 0, 2024-09-16T02:08:11.749 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-09-16T02:08:11.749 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 0, 2024-09-16T02:08:11.749 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 0, 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 0, 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.750 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.751 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.752 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.753 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.753 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-09-16T02:08:11.753 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.753 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.753 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.753 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.753 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-09-16T02:08:11.753 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.753 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.753 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.754 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.754 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-09-16T02:08:11.754 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.754 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.754 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.754 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.754 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-09-16T02:08:11.754 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.754 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.757 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.757 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.757 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-09-16T02:08:11.757 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.757 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.757 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.757 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.757 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.758 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-09-16T02:08:11.759 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.760 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.761 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.761 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-09-16T02:08:11.761 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.761 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.761 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.761 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.761 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-09-16T02:08:11.761 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.761 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.761 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.762 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-09-16T02:08:11.763 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.763 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.763 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.763 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.763 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-09-16T02:08:11.763 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.763 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.763 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.763 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.763 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-09-16T02:08:11.764 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.765 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.765 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.765 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:11.765 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.765 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-09-16T02:08:11.765 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 0, 2024-09-16T02:08:11.765 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 1, 2024-09-16T02:08:11.765 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 1, 2024-09-16T02:08:11.766 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 1, 2024-09-16T02:08:11.766 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-09-16T02:08:11.766 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 0, 2024-09-16T02:08:11.766 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 0, 2024-09-16T02:08:11.766 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 1328, 2024-09-16T02:08:11.766 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-09-16T02:08:11.766 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.766 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-16T02:08:11.766 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:08:11.766 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:08:11.767 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:08:11.768 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:08:11.768 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:08:11.768 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-16T02:08:11.768 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:08:11.768 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-16T02:08:11.768 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:08:11.768 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:08:11.768 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:08:11.768 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:08:11.768 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:08:11.769 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:08:11.769 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:08:11.769 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-16T02:08:11.769 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:08:11.769 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:08:11.769 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:08:11.769 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-16T02:08:11.769 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-16T02:08:11.769 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-16T02:08:11.769 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:08:11.770 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:08:11.770 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:08:11.770 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:08:11.770 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 181159, 2024-09-16T02:08:11.770 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 44, 2024-09-16T02:08:11.770 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:08:11.770 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-09-16T02:08:11.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-16T02:08:11.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-16T02:08:11.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:08:11.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-16T02:08:11.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:08:11.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-16T02:08:11.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 7984, 2024-09-16T02:08:11.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 165, 2024-09-16T02:08:11.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:08:11.772 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:08:11.772 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-16T02:08:11.772 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-16T02:08:11.772 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 8512, 2024-09-16T02:08:11.772 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 13, 2024-09-16T02:08:11.772 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-16T02:08:11.772 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-16T02:08:11.772 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-16T02:08:11.772 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-16T02:08:11.773 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.773 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-09-16T02:08:11.773 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-09-16T02:08:11.773 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-09-16T02:08:11.773 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 697, 2024-09-16T02:08:11.773 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 21, 2024-09-16T02:08:11.773 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 3, 2024-09-16T02:08:11.773 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 697, 2024-09-16T02:08:11.773 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-09-16T02:08:11.773 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 697, 2024-09-16T02:08:11.774 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.830979016, 2024-09-16T02:08:11.774 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001192222 2024-09-16T02:08:11.774 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.774 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-09-16T02:08:11.774 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-09-16T02:08:11.774 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 697, 2024-09-16T02:08:11.774 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 698 2024-09-16T02:08:11.774 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.774 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 697, 2024-09-16T02:08:11.774 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 697, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 0, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 426, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 0, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 269, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 0, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 0, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-09-16T02:08:11.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 1, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-09-16T02:08:11.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-09-16T02:08:11.777 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 81, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 4, 2024-09-16T02:08:11.778 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 4, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 4, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 0, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 2, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 0 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 0, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.779 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:08:11.780 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 800, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 867610, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 800, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 800, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 867610, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.781 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 697, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 1127, 2024-09-16T02:08:11.782 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 697, 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1127, 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:11.783 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 697, 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 697, 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 697, 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 697, 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.784 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x55bb7ac24b40": { 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-16T02:08:11.786 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-16T02:08:11.787 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-16T02:08:11.787 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:11.787 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:11.787 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:11.787 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:11.787 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:11.787 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:11.787 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-16T02:08:12.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:11 smithi103 ceph-mon[30398]: pgmap v422: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:08:12.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:11 smithi136 ceph-mon[35323]: pgmap v422: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:08:12.029 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:11 smithi172 ceph-mon[35340]: pgmap v422: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T02:08:12.475 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-16T02:08:12.475 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-16T02:08:12.475 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-16T02:08:12.475 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-16T02:08:12.475 DEBUG:tasks.check_counter:Getting stats from i 2024-09-16T02:08:12.476 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-16T02:08:12.721 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:08:13.203 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-16T02:08:13.204 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:08:13.204 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 483, 2024-09-16T02:08:13.204 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 390, 2024-09-16T02:08:13.204 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 864789, 2024-09-16T02:08:13.204 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 174956, 2024-09-16T02:08:13.204 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 7, 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 2, 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.045487565, 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.016100370, 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.033048262, 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.001975863, 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 391, 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.027685846, 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000070807 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.205 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 586, 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000103136, 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000176 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 751936, 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 55808 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 33, 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 95, 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 7860, 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 299118, 2024-09-16T02:08:13.206 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 15, 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 4, 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.010320309, 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.003470475, 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.005573608, 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000359119, 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 93, 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.013335805, 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000143395 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:13.207 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 23, 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000002651, 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000115 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 2336, 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 187936 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 28, 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 30, 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 9453, 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 32638, 2024-09-16T02:08:13.208 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 10, 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 5, 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.007674461, 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.001125468, 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.006187101, 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000219970, 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 27, 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.014423596, 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000534207 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.209 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 17, 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000004762, 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000280 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 4656, 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 1888 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-09-16T02:08:13.210 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 47, 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.002047969, 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000043573 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2, 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000015072, 2024-09-16T02:08:13.211 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000007536 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 7, 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 4, 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4, 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001948521, 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000487130 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.213 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 3, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 11, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 17, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 1, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 16, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 13, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 8, 2024-09-16T02:08:13.214 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 3, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 19, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 13, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 1, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 4, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-09-16T02:08:13.215 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 4, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 234, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-09-16T02:08:13.216 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 0, 2024-09-16T02:08:13.217 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 1 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 13, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 8, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 10, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-09-16T02:08:13.218 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-09-16T02:08:13.219 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 7, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 7, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 2, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-09-16T02:08:13.220 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 4196271, 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 0, 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 6, 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.012064922, 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002010820 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.221 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 0 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 21, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 27, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 6, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 20, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 25, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 5, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 17, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 25, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 8, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 4, 2024-09-16T02:08:13.222 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 4, 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 49864, 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 223520 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 11, 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 7, 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 11, 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 1, 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.223 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2, 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001475620, 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000737810 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.224 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2, 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000472901, 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000236450 2024-09-16T02:08:13.225 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.226 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.227 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.228 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.228 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.228 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-09-16T02:08:13.228 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.229 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-09-16T02:08:13.230 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.231 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-09-16T02:08:13.232 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.233 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.234 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-09-16T02:08:13.235 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 1, 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 3, 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 2, 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 1, 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 3, 2024-09-16T02:08:13.236 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 1, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 159, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:08:13.237 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:08:13.238 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:08:13.239 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 231561, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 78, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 88, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 1, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 7984, 2024-09-16T02:08:13.240 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 165, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 85221, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 348, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.241 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 44, 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 12494, 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 28, 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 44, 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 44, 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.223994341, 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.005090780 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.242 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-09-16T02:08:13.243 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-09-16T02:08:13.243 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 44, 2024-09-16T02:08:13.243 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 68 2024-09-16T02:08:13.243 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.243 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 44, 2024-09-16T02:08:13.243 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 0, 2024-09-16T02:08:13.243 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 44, 2024-09-16T02:08:13.243 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-09-16T02:08:13.243 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-09-16T02:08:13.243 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 0, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 10, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 0, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 6, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 7, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 6, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-09-16T02:08:13.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 20, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-09-16T02:08:13.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-09-16T02:08:13.246 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 81, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 4, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 8, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 8, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 17, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 0, 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 2 2024-09-16T02:08:13.247 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 1, 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T02:08:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 540, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 823531, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 540, 2024-09-16T02:08:13.249 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 540, 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 823531, 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-16T02:08:13.250 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 44, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 12494, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 44, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 12494, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.251 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 44, 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 44, 2024-09-16T02:08:13.252 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 44, 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 44, 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-16T02:08:13.253 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:13.254 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x55978ac26960": { 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 7, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 1967, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 7, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 6, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1967, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:13.255 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:13.256 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:13.256 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:13.256 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:13.256 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-16T02:08:13.795 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:13 smithi172 ceph-mon[35340]: pgmap v423: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T02:08:13.965 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-16T02:08:13.966 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-16T02:08:13.966 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-16T02:08:13.966 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=13 2024-09-16T02:08:13.966 DEBUG:tasks.check_counter:Getting stats from l 2024-09-16T02:08:13.966 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-16T02:08:14.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:13 smithi103 ceph-mon[30398]: pgmap v423: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T02:08:14.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:13 smithi136 ceph-mon[35323]: pgmap v423: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-16T02:08:14.210 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T02:08:14.717 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-16T02:08:14.717 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T02:08:14.717 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 4, 2024-09-16T02:08:14.717 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 4, 2024-09-16T02:08:14.717 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 3072, 2024-09-16T02:08:14.717 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 448, 2024-09-16T02:08:14.717 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 0, 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.002936590, 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.000034338, 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.002098894, 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3, 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.003940777, 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001313592 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:14.718 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1, 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 68, 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 160, 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 90560, 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 3, 2024-09-16T02:08:14.719 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 2, 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.004270518, 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.002881879, 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.000529387, 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 66, 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.004134060, 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000062637 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:14.720 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 90560 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 95, 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 90, 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 720576, 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 54112, 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 2, 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 1, 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.018303018, 2024-09-16T02:08:14.721 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.006022194, 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.019455401, 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 89, 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.006372514, 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000071601 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.722 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 720576, 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 54112 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 0, 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-16T02:08:14.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T02:08:14.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-16T02:08:14.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 148841, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 28, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-16T02:08:14.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 0, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 0, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 928, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 2, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T02:08:14.727 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 96, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 707188, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 96, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 96, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 707188, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-16T02:08:14.728 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-16T02:08:14.729 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-16T02:08:14.729 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:14.729 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-16T02:08:14.729 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-16T02:08:15.538 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-16T02:08:15.538 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-16T02:08:15.538 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-16T02:08:15.538 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-16T02:08:15.539 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-16T02:08:15.549 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-16T02:08:15.549 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T02:08:15.549 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T02:08:15.591 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-16T02:08:15.591 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T02:08:15.671 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T02:08:16.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:15 smithi103 ceph-mon[30398]: pgmap v424: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:16.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:15 smithi136 ceph-mon[35323]: pgmap v424: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:16.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:15 smithi172 ceph-mon[35340]: pgmap v424: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T02:08:16.088 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:08:16.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.599+0000 7fbb6ac00640 1 -- 172.21.15.103:0/2555442208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb64102790 msgr2=0x7fbb641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:16.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.599+0000 7fbb6ac00640 1 --2- 172.21.15.103:0/2555442208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb64102790 0x7fbb641067e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009900 tx=0x7fbb5002a3f0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.599+0000 7fbb6ac00640 1 -- 172.21.15.103:0/2555442208 shutdown_connections 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.599+0000 7fbb6ac00640 1 --2- 172.21.15.103:0/2555442208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb64106f10 0x7fbb6410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.599+0000 7fbb6ac00640 1 --2- 172.21.15.103:0/2555442208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb64102790 0x7fbb641067e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.599+0000 7fbb6ac00640 1 --2- 172.21.15.103:0/2555442208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb64101de0 0x7fbb641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.599+0000 7fbb6ac00640 1 -- 172.21.15.103:0/2555442208 >> 172.21.15.103:0/2555442208 conn(0x7fbb640fd8c0 msgr2=0x7fbb640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.599+0000 7fbb6ac00640 1 -- 172.21.15.103:0/2555442208 shutdown_connections 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.599+0000 7fbb6ac00640 1 -- 172.21.15.103:0/2555442208 wait complete. 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.600+0000 7fbb6ac00640 1 Processor -- start 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.600+0000 7fbb6ac00640 1 -- start start 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.600+0000 7fbb6ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb64101de0 0x7fbb640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb6ac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb64102790 0x7fbb64074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb6ac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb64106f10 0x7fbb64074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb6ac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64111f70 con 0x7fbb64106f10 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb6ac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64111df0 con 0x7fbb64102790 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb6ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641120f0 con 0x7fbb64101de0 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb68c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb64106f10 0x7fbb64074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb68c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb64106f10 0x7fbb64074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51264/0 (socket says 172.21.15.103:51264) 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb68c00640 1 -- 172.21.15.103:0/1320634963 learned_addr learned my addr 172.21.15.103:0/1320634963 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb63e00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb64101de0 0x7fbb640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb63400640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb64102790 0x7fbb64074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb68c00640 1 -- 172.21.15.103:0/1320634963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb64101de0 msgr2=0x7fbb640744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb68c00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb64101de0 0x7fbb640744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb68c00640 1 -- 172.21.15.103:0/1320634963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb64102790 msgr2=0x7fbb64074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb68c00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb64102790 0x7fbb64074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb68c00640 1 -- 172.21.15.103:0/1320634963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb641a68b0 con 0x7fbb64106f10 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.601+0000 7fbb63e00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb64101de0 0x7fbb640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.602+0000 7fbb68c00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb64106f10 0x7fbb64074f70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00eb10 tx=0x7fbb5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.602+0000 7fbb60c00640 1 -- 172.21.15.103:0/1320634963 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c004b20 con 0x7fbb64106f10 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.602+0000 7fbb60c00640 1 -- 172.21.15.103:0/1320634963 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb5c004cc0 con 0x7fbb64106f10 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.602+0000 7fbb6ac00640 1 -- 172.21.15.103:0/1320634963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb641a6ba0 con 0x7fbb64106f10 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.602+0000 7fbb60c00640 1 -- 172.21.15.103:0/1320634963 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c010760 con 0x7fbb64106f10 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.602+0000 7fbb6ac00640 1 -- 172.21.15.103:0/1320634963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb641a7100 con 0x7fbb64106f10 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.603+0000 7fbb6ac00640 1 -- 172.21.15.103:0/1320634963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb641021c0 con 0x7fbb64106f10 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.605+0000 7fbb60c00640 1 -- 172.21.15.103:0/1320634963 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fbb5c005020 con 0x7fbb64106f10 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.606+0000 7fbb60c00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbb480a32c0 0x7fbb480a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.606+0000 7fbb60c00640 1 -- 172.21.15.103:0/1320634963 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fbb5c0cffc0 con 0x7fbb64106f10 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.607+0000 7fbb63e00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbb480a32c0 0x7fbb480a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.607+0000 7fbb63e00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbb480a32c0 0x7fbb480a5780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009fc0 tx=0x7fbb54005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:16.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.610+0000 7fbb60c00640 1 -- 172.21.15.103:0/1320634963 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb5c09cc00 con 0x7fbb64106f10 2024-09-16T02:08:16.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.799+0000 7fbb6ac00640 1 -- 172.21.15.103:0/1320634963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb6410a0f0 con 0x7fbb64106f10 2024-09-16T02:08:16.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.800+0000 7fbb60c00640 1 -- 172.21.15.103:0/1320634963 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) ==== 81+0+600 (secure 0 0 0) 0x7fbb5c09c500 con 0x7fbb64106f10 2024-09-16T02:08:16.802 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1646778488 2024-09-17T01:56:28.359910+0000 2024-09-16T02:08:16.802 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1801195624 2024-09-17T01:56:28.359910+0000 2024-09-16T02:08:16.802 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1537689916 2024-09-17T01:56:28.359910+0000 2024-09-16T02:08:16.803 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/623307529 2024-09-17T01:56:28.359910+0000 2024-09-16T02:08:16.803 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/1247798496 2024-09-17T01:56:01.936729+0000 2024-09-16T02:08:16.803 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/623307529 2024-09-17T01:56:28.359910+0000 2024-09-16T02:08:16.803 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/1247798496 2024-09-17T01:56:01.936729+0000 2024-09-16T02:08:16.803 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2568354020 2024-09-17T01:56:01.936729+0000 2024-09-16T02:08:16.803 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1667549073 2024-09-17T01:56:01.936729+0000 2024-09-16T02:08:16.803 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3586311435 2024-09-17T01:56:01.936729+0000 2024-09-16T02:08:16.803 INFO:teuthology.orchestra.run.smithi103.stderr:listed 10 entries 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.802+0000 7fbb6ac00640 1 -- 172.21.15.103:0/1320634963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbb480a32c0 msgr2=0x7fbb480a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.802+0000 7fbb6ac00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbb480a32c0 0x7fbb480a5780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009fc0 tx=0x7fbb54005f90 comp rx=0 tx=0).stop 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.802+0000 7fbb6ac00640 1 -- 172.21.15.103:0/1320634963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb64106f10 msgr2=0x7fbb64074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.802+0000 7fbb6ac00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb64106f10 0x7fbb64074f70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00eb10 tx=0x7fbb5c00c2d0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.802+0000 7fbb6ac00640 1 -- 172.21.15.103:0/1320634963 shutdown_connections 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.802+0000 7fbb6ac00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbb480a32c0 0x7fbb480a5780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.802+0000 7fbb6ac00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb64106f10 0x7fbb64074f70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.802+0000 7fbb6ac00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb64102790 0x7fbb64074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.802+0000 7fbb6ac00640 1 --2- 172.21.15.103:0/1320634963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb64101de0 0x7fbb640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.802+0000 7fbb6ac00640 1 -- 172.21.15.103:0/1320634963 >> 172.21.15.103:0/1320634963 conn(0x7fbb640fd8c0 msgr2=0x7fbb640fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:16.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.803+0000 7fbb6ac00640 1 -- 172.21.15.103:0/1320634963 shutdown_connections 2024-09-16T02:08:16.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:16.803+0000 7fbb6ac00640 1 -- 172.21.15.103:0/1320634963 wait complete. 2024-09-16T02:08:17.565 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T02:08:17.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:08:17.566 DEBUG:teuthology.orchestra.run.smithi103:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T02:08:17.649 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2024-09-16T02:08:17.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:08:17.650 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T02:08:17.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:08:17.713 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-16T02:08:17.749 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T02:08:17.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:08:17.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T02:08:17.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:17 smithi103 ceph-mon[30398]: pgmap v425: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:08:17.838 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:17 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1320634963' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T02:08:17.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T02:08:17.849 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-09-16T02:08:18.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:17 smithi136 ceph-mon[35323]: pgmap v425: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:08:18.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:17 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1320634963' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T02:08:18.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:17 smithi172 ceph-mon[35340]: pgmap v425: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T02:08:18.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:17 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1320634963' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T02:08:18.167 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T02:08:18.177 INFO:tasks.cephadm:Teardown begin 2024-09-16T02:08:18.178 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T02:08:18.178 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:08:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:08:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.285+0000 7efd3fe00640 1 -- 172.21.15.103:0/793842259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd40074510 msgr2=0x7efd400748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.285+0000 7efd3fe00640 1 --2- 172.21.15.103:0/793842259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd40074510 0x7efd400748f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efd30009900 tx=0x7efd3002a480 comp rx=0 tx=0).stop 2024-09-16T02:08:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.286+0000 7efd3fe00640 1 -- 172.21.15.103:0/793842259 shutdown_connections 2024-09-16T02:08:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.286+0000 7efd3fe00640 1 --2- 172.21.15.103:0/793842259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd4010e280 0x7efd40110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.286+0000 7efd3fe00640 1 --2- 172.21.15.103:0/793842259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd40074e30 0x7efd4010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.286+0000 7efd3fe00640 1 --2- 172.21.15.103:0/793842259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd40074510 0x7efd400748f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.286+0000 7efd3fe00640 1 -- 172.21.15.103:0/793842259 >> 172.21.15.103:0/793842259 conn(0x7efd400fd740 msgr2=0x7efd400ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.286+0000 7efd3fe00640 1 -- 172.21.15.103:0/793842259 shutdown_connections 2024-09-16T02:08:19.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.286+0000 7efd3fe00640 1 -- 172.21.15.103:0/793842259 wait complete. 2024-09-16T02:08:19.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.287+0000 7efd3fe00640 1 Processor -- start 2024-09-16T02:08:19.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.287+0000 7efd3fe00640 1 -- start start 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.287+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd40074510 0x7efd4019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.287+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd40074e30 0x7efd4019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.287+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd4010e280 0x7efd4019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efd40114000 con 0x7efd40074510 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd40113e80 con 0x7efd4010e280 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd40114180 con 0x7efd40074e30 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd40074510 0x7efd4019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd40074510 0x7efd4019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57990/0 (socket says 172.21.15.103:57990) 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3ea00640 1 -- 172.21.15.103:0/226992047 learned_addr learned my addr 172.21.15.103:0/226992047 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3e000640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd40074e30 0x7efd4019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:19.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3f400640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd4010e280 0x7efd4019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:19.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3ea00640 1 -- 172.21.15.103:0/226992047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd40074e30 msgr2=0x7efd4019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:19.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3ea00640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd40074e30 0x7efd4019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:19.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3ea00640 1 -- 172.21.15.103:0/226992047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd4010e280 msgr2=0x7efd4019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:19.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3ea00640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd4010e280 0x7efd4019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:19.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.288+0000 7efd3ea00640 1 -- 172.21.15.103:0/226992047 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4019f450 con 0x7efd40074510 2024-09-16T02:08:19.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.289+0000 7efd3ea00640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd40074510 0x7efd4019a4a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efd30004770 tx=0x7efd30003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:19.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.289+0000 7efd37400640 1 -- 172.21.15.103:0/226992047 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3003f070 con 0x7efd40074510 2024-09-16T02:08:19.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.289+0000 7efd37400640 1 -- 172.21.15.103:0/226992047 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd30031ca0 con 0x7efd40074510 2024-09-16T02:08:19.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.289+0000 7efd3fe00640 1 -- 172.21.15.103:0/226992047 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4019f6e0 con 0x7efd40074510 2024-09-16T02:08:19.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.289+0000 7efd37400640 1 -- 172.21.15.103:0/226992047 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30035030 con 0x7efd40074510 2024-09-16T02:08:19.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.289+0000 7efd3fe00640 1 -- 172.21.15.103:0/226992047 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd401a7020 con 0x7efd40074510 2024-09-16T02:08:19.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.290+0000 7efd3fe00640 1 -- 172.21.15.103:0/226992047 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd40075c50 con 0x7efd40074510 2024-09-16T02:08:19.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.294+0000 7efd37400640 1 -- 172.21.15.103:0/226992047 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7efd30042070 con 0x7efd40074510 2024-09-16T02:08:19.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.296+0000 7efd37400640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd240a3240 0x7efd240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:19.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.296+0000 7efd37400640 1 -- 172.21.15.103:0/226992047 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7efd300f0070 con 0x7efd40074510 2024-09-16T02:08:19.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.296+0000 7efd3e000640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd240a3240 0x7efd240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:19.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.297+0000 7efd3e000640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd240a3240 0x7efd240a5700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efd4010f610 tx=0x7efd2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:19.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.299+0000 7efd37400640 1 -- 172.21.15.103:0/226992047 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd300bc660 con 0x7efd40074510 2024-09-16T02:08:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.519+0000 7efd3fe00640 1 -- 172.21.15.103:0/226992047 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd40109400 con 0x7efd40074510 2024-09-16T02:08:19.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.520+0000 7efd37400640 1 -- 172.21.15.103:0/226992047 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7efd300c0960 con 0x7efd40074510 2024-09-16T02:08:19.521 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:08:19.522 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":24,"btime":"2024-09-16T02:05:32:550231+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:05:31.661572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24506,24505]},"id":1}]} 2024-09-16T02:08:19.522 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 24 2024-09-16T02:08:19.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 -- 172.21.15.103:0/226992047 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd240a3240 msgr2=0x7efd240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:19.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd240a3240 0x7efd240a5700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efd4010f610 tx=0x7efd2c005f90 comp rx=0 tx=0).stop 2024-09-16T02:08:19.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 -- 172.21.15.103:0/226992047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd40074510 msgr2=0x7efd4019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:19.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd40074510 0x7efd4019a4a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efd30004770 tx=0x7efd30003c10 comp rx=0 tx=0).stop 2024-09-16T02:08:19.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 -- 172.21.15.103:0/226992047 shutdown_connections 2024-09-16T02:08:19.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7efd240a3240 0x7efd240a5700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:19.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd4010e280 0x7efd4019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:19.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd40074e30 0x7efd4019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:19.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 --2- 172.21.15.103:0/226992047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd40074510 0x7efd4019a4a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:19.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 -- 172.21.15.103:0/226992047 >> 172.21.15.103:0/226992047 conn(0x7efd400fd740 msgr2=0x7efd400fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:19.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 -- 172.21.15.103:0/226992047 shutdown_connections 2024-09-16T02:08:19.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:19.523+0000 7efd3fe00640 1 -- 172.21.15.103:0/226992047 wait complete. 2024-09-16T02:08:19.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:19 smithi103 ceph-mon[30398]: pgmap v426: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T02:08:19.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:19 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/226992047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:08:20.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:19 smithi136 ceph-mon[35323]: pgmap v426: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T02:08:20.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:19 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/226992047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:08:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:19 smithi172 ceph-mon[35340]: pgmap v426: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T02:08:20.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:19 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/226992047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:08:20.325 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T02:08:20.700 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:08:21.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.234+0000 7fbdfae00640 1 -- 172.21.15.103:0/3592135438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdf41047e0 msgr2=0x7fbdf4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:21.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.234+0000 7fbdfae00640 1 --2- 172.21.15.103:0/3592135438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdf41047e0 0x7fbdf4108830 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009900 tx=0x7fbde002a410 comp rx=0 tx=0).stop 2024-09-16T02:08:21.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.234+0000 7fbdfae00640 1 -- 172.21.15.103:0/3592135438 shutdown_connections 2024-09-16T02:08:21.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.234+0000 7fbdfae00640 1 --2- 172.21.15.103:0/3592135438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf4108f60 0x7fbdf4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.234+0000 7fbdfae00640 1 --2- 172.21.15.103:0/3592135438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdf41047e0 0x7fbdf4108830 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.234+0000 7fbdfae00640 1 --2- 172.21.15.103:0/3592135438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbdf4103e30 0x7fbdf4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.234+0000 7fbdfae00640 1 -- 172.21.15.103:0/3592135438 >> 172.21.15.103:0/3592135438 conn(0x7fbdf40ff8d0 msgr2=0x7fbdf4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:21.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.234+0000 7fbdfae00640 1 -- 172.21.15.103:0/3592135438 shutdown_connections 2024-09-16T02:08:21.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.234+0000 7fbdfae00640 1 -- 172.21.15.103:0/3592135438 wait complete. 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.235+0000 7fbdfae00640 1 Processor -- start 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.235+0000 7fbdfae00640 1 -- start start 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdfae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdf4103e30 0x7fbdf419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdfae00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbdf41047e0 0x7fbdf419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdfae00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf4108f60 0x7fbdf419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdfae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf4113fc0 con 0x7fbdf4103e30 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdfae00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf4113e40 con 0x7fbdf41047e0 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdfae00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf4114140 con 0x7fbdf4108f60 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdf9a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdf4103e30 0x7fbdf419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdf9a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdf4103e30 0x7fbdf419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58006/0 (socket says 172.21.15.103:58006) 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdf9a00640 1 -- 172.21.15.103:0/1474308215 learned_addr learned my addr 172.21.15.103:0/1474308215 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:08:21.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdfa400640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf4108f60 0x7fbdf419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:21.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdf9000640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbdf41047e0 0x7fbdf419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:21.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdf9a00640 1 -- 172.21.15.103:0/1474308215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf4108f60 msgr2=0x7fbdf419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:21.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.236+0000 7fbdf9a00640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf4108f60 0x7fbdf419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.237+0000 7fbdf9a00640 1 -- 172.21.15.103:0/1474308215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbdf41047e0 msgr2=0x7fbdf419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:21.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.237+0000 7fbdf9a00640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbdf41047e0 0x7fbdf419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.237+0000 7fbdf9a00640 1 -- 172.21.15.103:0/1474308215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf419f410 con 0x7fbdf4103e30 2024-09-16T02:08:21.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.237+0000 7fbdf9a00640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdf4103e30 0x7fbdf419a460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbde800ebc0 tx=0x7fbde800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:21.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.237+0000 7fbdf2000640 1 -- 172.21.15.103:0/1474308215 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde8010630 con 0x7fbdf4103e30 2024-09-16T02:08:21.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.237+0000 7fbdf2000640 1 -- 172.21.15.103:0/1474308215 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbde8010cb0 con 0x7fbdf4103e30 2024-09-16T02:08:21.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.237+0000 7fbdf2000640 1 -- 172.21.15.103:0/1474308215 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde8018bd0 con 0x7fbdf4103e30 2024-09-16T02:08:21.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.237+0000 7fbdfae00640 1 -- 172.21.15.103:0/1474308215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf419f700 con 0x7fbdf4103e30 2024-09-16T02:08:21.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.237+0000 7fbdfae00640 1 -- 172.21.15.103:0/1474308215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf41a6fe0 con 0x7fbdf4103e30 2024-09-16T02:08:21.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.238+0000 7fbdfae00640 1 -- 172.21.15.103:0/1474308215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf4105310 con 0x7fbdf4103e30 2024-09-16T02:08:21.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.240+0000 7fbdf2000640 1 -- 172.21.15.103:0/1474308215 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fbde80107d0 con 0x7fbdf4103e30 2024-09-16T02:08:21.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.241+0000 7fbdf2000640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbddc0a32c0 0x7fbddc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:21.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.242+0000 7fbdf2000640 1 -- 172.21.15.103:0/1474308215 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fbde80cff90 con 0x7fbdf4103e30 2024-09-16T02:08:21.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.242+0000 7fbdf9000640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbddc0a32c0 0x7fbddc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:21.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.243+0000 7fbdf9000640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbddc0a32c0 0x7fbddc0a5780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009230 tx=0x7fbde00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:21.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.245+0000 7fbdf2000640 1 -- 172.21.15.103:0/1474308215 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbde809c580 con 0x7fbdf4103e30 2024-09-16T02:08:21.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.454+0000 7fbdfae00640 1 -- 172.21.15.103:0/1474308215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdf4104210 con 0x7fbdf4103e30 2024-09-16T02:08:21.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.455+0000 7fbdf2000640 1 -- 172.21.15.103:0/1474308215 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7fbde80a0880 con 0x7fbdf4103e30 2024-09-16T02:08:21.456 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:08:21.456 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":24,"btime":"2024-09-16T02:05:32:550231+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2357019382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2357019382},{"type":"v1","addr":"172.21.15.103:6835","nonce":2357019382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3043303715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3043303715},{"type":"v1","addr":"172.21.15.103:6839","nonce":3043303715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/2552214171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":2552214171},{"type":"v1","addr":"172.21.15.103:6841","nonce":2552214171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1362058923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1362058923},{"type":"v1","addr":"172.21.15.172:6833","nonce":1362058923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/3604147096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3604147096},{"type":"v1","addr":"172.21.15.136:6833","nonce":3604147096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2740838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2740838933},{"type":"v1","addr":"172.21.15.103:6837","nonce":2740838933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T02:03:37.830568+0000","modified":"2024-09-16T02:05:31.661572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24506,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/2566717089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2566717089},{"type":"v1","addr":"172.21.15.136:6835","nonce":2566717089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2720370264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2720370264},{"type":"v1","addr":"172.21.15.172:6835","nonce":2720370264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6837/1584538248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":1584538248},{"type":"v1","addr":"172.21.15.172:6837","nonce":1584538248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6839/3474994563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3474994563},{"type":"v1","addr":"172.21.15.172:6839","nonce":3474994563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6837/236571207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":236571207},{"type":"v1","addr":"172.21.15.136:6837","nonce":236571207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/3041647128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":3041647128},{"type":"v1","addr":"172.21.15.136:6839","nonce":3041647128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24506,24505]},"id":1}]} 2024-09-16T02:08:21.457 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 24 2024-09-16T02:08:21.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 -- 172.21.15.103:0/1474308215 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbddc0a32c0 msgr2=0x7fbddc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:21.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbddc0a32c0 0x7fbddc0a5780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009230 tx=0x7fbde00058d0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 -- 172.21.15.103:0/1474308215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdf4103e30 msgr2=0x7fbdf419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:21.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdf4103e30 0x7fbdf419a460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbde800ebc0 tx=0x7fbde800c6a0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 -- 172.21.15.103:0/1474308215 shutdown_connections 2024-09-16T02:08:21.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fbddc0a32c0 0x7fbddc0a5780 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf4108f60 0x7fbdf419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbdf41047e0 0x7fbdf419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 --2- 172.21.15.103:0/1474308215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdf4103e30 0x7fbdf419a460 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:21.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 -- 172.21.15.103:0/1474308215 >> 172.21.15.103:0/1474308215 conn(0x7fbdf40ff8d0 msgr2=0x7fbdf4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:21.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 -- 172.21.15.103:0/1474308215 shutdown_connections 2024-09-16T02:08:21.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:21.458+0000 7fbdfae00640 1 -- 172.21.15.103:0/1474308215 wait complete. 2024-09-16T02:08:21.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:21 smithi103 ceph-mon[30398]: pgmap v427: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2024-09-16T02:08:21.755 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:21 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/1474308215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:08:22.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:21 smithi136 ceph-mon[35323]: pgmap v427: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2024-09-16T02:08:22.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:21 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/1474308215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:08:22.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:21 smithi172 ceph-mon[35340]: pgmap v427: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2024-09-16T02:08:22.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:21 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/1474308215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T02:08:22.177 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T02:08:22.552 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:08:23.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.090+0000 7fa9da200640 1 -- 172.21.15.103:0/658871187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9d4109120 msgr2=0x7fa9d4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:23.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.090+0000 7fa9da200640 1 --2- 172.21.15.103:0/658871187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9d4109120 0x7fa9d4110ca0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009ca0 tx=0x7fa9cc02a8c0 comp rx=0 tx=0).stop 2024-09-16T02:08:23.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.091+0000 7fa9da200640 1 -- 172.21.15.103:0/658871187 shutdown_connections 2024-09-16T02:08:23.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.091+0000 7fa9da200640 1 --2- 172.21.15.103:0/658871187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9d4109120 0x7fa9d4110ca0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:23.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.091+0000 7fa9da200640 1 --2- 172.21.15.103:0/658871187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d41049a0 0x7fa9d41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:23.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.091+0000 7fa9da200640 1 --2- 172.21.15.103:0/658871187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9d4103ff0 0x7fa9d41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:23.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.091+0000 7fa9da200640 1 -- 172.21.15.103:0/658871187 >> 172.21.15.103:0/658871187 conn(0x7fa9d40ffa90 msgr2=0x7fa9d4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:23.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.091+0000 7fa9da200640 1 -- 172.21.15.103:0/658871187 shutdown_connections 2024-09-16T02:08:23.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.091+0000 7fa9da200640 1 -- 172.21.15.103:0/658871187 wait complete. 2024-09-16T02:08:23.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.092+0000 7fa9da200640 1 Processor -- start 2024-09-16T02:08:23.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.092+0000 7fa9da200640 1 -- start start 2024-09-16T02:08:23.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9da200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d4103ff0 0x7fa9d419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:23.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9da200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9d41049a0 0x7fa9d419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:23.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9da200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9d4109120 0x7fa9d419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:23.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9da200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4114180 con 0x7fa9d41049a0 2024-09-16T02:08:23.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9da200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4114000 con 0x7fa9d4109120 2024-09-16T02:08:23.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9da200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4114300 con 0x7fa9d4103ff0 2024-09-16T02:08:23.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9caa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9d41049a0 0x7fa9d419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:23.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9d3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d4103ff0 0x7fa9d419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:23.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9caa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9d41049a0 0x7fa9d419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58030/0 (socket says 172.21.15.103:58030) 2024-09-16T02:08:23.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9caa00640 1 -- 172.21.15.103:0/57871141 learned_addr learned my addr 172.21.15.103:0/57871141 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:08:23.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9d3e00640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9d4109120 0x7fa9d419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:23.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9caa00640 1 -- 172.21.15.103:0/57871141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d4103ff0 msgr2=0x7fa9d419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:23.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9caa00640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d4103ff0 0x7fa9d419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:23.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9caa00640 1 -- 172.21.15.103:0/57871141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9d4109120 msgr2=0x7fa9d419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:23.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9caa00640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9d4109120 0x7fa9d419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:23.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.093+0000 7fa9caa00640 1 -- 172.21.15.103:0/57871141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d419f570 con 0x7fa9d41049a0 2024-09-16T02:08:23.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.094+0000 7fa9caa00640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9d41049a0 0x7fa9d419ab00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00d950 tx=0x7fa9bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:23.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.094+0000 7fa9d0c00640 1 -- 172.21.15.103:0/57871141 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc017030 con 0x7fa9d41049a0 2024-09-16T02:08:23.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.094+0000 7fa9d0c00640 1 -- 172.21.15.103:0/57871141 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9bc004020 con 0x7fa9d41049a0 2024-09-16T02:08:23.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.094+0000 7fa9d0c00640 1 -- 172.21.15.103:0/57871141 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0047a0 con 0x7fa9d41049a0 2024-09-16T02:08:23.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.094+0000 7fa9da200640 1 -- 172.21.15.103:0/57871141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d419f860 con 0x7fa9d41049a0 2024-09-16T02:08:23.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.094+0000 7fa9da200640 1 -- 172.21.15.103:0/57871141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d41a7140 con 0x7fa9d41049a0 2024-09-16T02:08:23.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.095+0000 7fa9da200640 1 -- 172.21.15.103:0/57871141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d41043d0 con 0x7fa9d41049a0 2024-09-16T02:08:23.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.097+0000 7fa9d0c00640 1 -- 172.21.15.103:0/57871141 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa9bc0041d0 con 0x7fa9d41049a0 2024-09-16T02:08:23.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.099+0000 7fa9d0c00640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa9b80a32c0 0x7fa9b80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:23.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.099+0000 7fa9d0c00640 1 -- 172.21.15.103:0/57871141 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fa9bc0cf250 con 0x7fa9d41049a0 2024-09-16T02:08:23.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.100+0000 7fa9d3400640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa9b80a32c0 0x7fa9b80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:23.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.100+0000 7fa9d3400640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa9b80a32c0 0x7fa9b80a5780 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009bd0 tx=0x7fa9c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:23.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.103+0000 7fa9d0c00640 1 -- 172.21.15.103:0/57871141 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9bc09be90 con 0x7fa9d41049a0 2024-09-16T02:08:23.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.283+0000 7fa9da200640 1 -- 172.21.15.103:0/57871141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9d41a74d0 con 0x7fa9d41049a0 2024-09-16T02:08:23.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.284+0000 7fa9d0c00640 1 -- 172.21.15.103:0/57871141 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23048 (secure 0 0 0) 0x7fa9bc09b790 con 0x7fa9d41049a0 2024-09-16T02:08:23.286 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-16T02:08:23.286 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":81,"fsid":"a20dd7a4-73ce-11ef-bceb-c7b262605968","created":"2024-09-16T01:55:37.667040+0000","modified":"2024-09-16T02:05:30.540044+0000","last_up_change":"2024-09-16T02:02:43.569608+0000","last_in_change":"2024-09-16T02:02:19.998119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:58:50.953500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T02:03:33.662251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T02:03:36.012448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"839e2a76-348d-4d7f-b43b-f446e829aade","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6803","nonce":484703095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6805","nonce":484703095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6809","nonce":484703095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":484703095},{"type":"v1","addr":"172.21.15.103:6807","nonce":484703095}]},"public_addr":"172.21.15.103:6803/484703095","cluster_addr":"172.21.15.103:6805/484703095","heartbeat_back_addr":"172.21.15.103:6809/484703095","heartbeat_front_addr":"172.21.15.103:6807/484703095","state":["exists","up"]},{"osd":1,"uuid":"393999bb-cc86-41ed-b0fa-7ae957c11374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6801","nonce":1430190507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6803","nonce":1430190507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6807","nonce":1430190507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1430190507},{"type":"v1","addr":"172.21.15.136:6805","nonce":1430190507}]},"public_addr":"172.21.15.136:6801/1430190507","cluster_addr":"172.21.15.136:6803/1430190507","heartbeat_back_addr":"172.21.15.136:6807/1430190507","heartbeat_front_addr":"172.21.15.136:6805/1430190507","state":["exists","up"]},{"osd":2,"uuid":"a337ce9d-76c5-4410-b843-fce557c1401e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6801","nonce":544248973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6803","nonce":544248973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6807","nonce":544248973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":544248973},{"type":"v1","addr":"172.21.15.172:6805","nonce":544248973}]},"public_addr":"172.21.15.172:6801/544248973","cluster_addr":"172.21.15.172:6803/544248973","heartbeat_back_addr":"172.21.15.172:6807/544248973","heartbeat_front_addr":"172.21.15.172:6805/544248973","state":["exists","up"]},{"osd":3,"uuid":"c2735e5b-dd14-483d-84de-a0172d862b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6811","nonce":2129636883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6813","nonce":2129636883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6817","nonce":2129636883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2129636883},{"type":"v1","addr":"172.21.15.103:6815","nonce":2129636883}]},"public_addr":"172.21.15.103:6811/2129636883","cluster_addr":"172.21.15.103:6813/2129636883","heartbeat_back_addr":"172.21.15.103:6817/2129636883","heartbeat_front_addr":"172.21.15.103:6815/2129636883","state":["exists","up"]},{"osd":4,"uuid":"35714991-3234-4719-b05f-604de810aac7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6809","nonce":2555490296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6811","nonce":2555490296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6815","nonce":2555490296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2555490296},{"type":"v1","addr":"172.21.15.136:6813","nonce":2555490296}]},"public_addr":"172.21.15.136:6809/2555490296","cluster_addr":"172.21.15.136:6811/2555490296","heartbeat_back_addr":"172.21.15.136:6815/2555490296","heartbeat_front_addr":"172.21.15.136:6813/2555490296","state":["exists","up"]},{"osd":5,"uuid":"f17f3197-8dc3-4e97-b7ee-a36975327aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6809","nonce":1913335537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6811","nonce":1913335537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6815","nonce":1913335537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1913335537},{"type":"v1","addr":"172.21.15.172:6813","nonce":1913335537}]},"public_addr":"172.21.15.172:6809/1913335537","cluster_addr":"172.21.15.172:6811/1913335537","heartbeat_back_addr":"172.21.15.172:6815/1913335537","heartbeat_front_addr":"172.21.15.172:6813/1913335537","state":["exists","up"]},{"osd":6,"uuid":"8ec56a5c-6c9b-473f-9ca2-084dc6d81ccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6819","nonce":1754743602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6821","nonce":1754743602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6825","nonce":1754743602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1754743602},{"type":"v1","addr":"172.21.15.103:6823","nonce":1754743602}]},"public_addr":"172.21.15.103:6819/1754743602","cluster_addr":"172.21.15.103:6821/1754743602","heartbeat_back_addr":"172.21.15.103:6825/1754743602","heartbeat_front_addr":"172.21.15.103:6823/1754743602","state":["exists","up"]},{"osd":7,"uuid":"a8c4b067-a2b3-404e-867e-8c8e51ecd4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6817","nonce":2248125128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6819","nonce":2248125128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6823","nonce":2248125128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2248125128},{"type":"v1","addr":"172.21.15.136:6821","nonce":2248125128}]},"public_addr":"172.21.15.136:6817/2248125128","cluster_addr":"172.21.15.136:6819/2248125128","heartbeat_back_addr":"172.21.15.136:6823/2248125128","heartbeat_front_addr":"172.21.15.136:6821/2248125128","state":["exists","up"]},{"osd":8,"uuid":"b835bf50-420f-42f9-8ca5-c3f1527be672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6817","nonce":1611593335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6819","nonce":1611593335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6823","nonce":1611593335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1611593335},{"type":"v1","addr":"172.21.15.172:6821","nonce":1611593335}]},"public_addr":"172.21.15.172:6817/1611593335","cluster_addr":"172.21.15.172:6819/1611593335","heartbeat_back_addr":"172.21.15.172:6823/1611593335","heartbeat_front_addr":"172.21.15.172:6821/1611593335","state":["exists","up"]},{"osd":9,"uuid":"46691205-fd3d-4f55-92bf-eae5bc09f6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6827","nonce":1291384117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6829","nonce":1291384117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6833","nonce":1291384117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1291384117},{"type":"v1","addr":"172.21.15.103:6831","nonce":1291384117}]},"public_addr":"172.21.15.103:6827/1291384117","cluster_addr":"172.21.15.103:6829/1291384117","heartbeat_back_addr":"172.21.15.103:6833/1291384117","heartbeat_front_addr":"172.21.15.103:6831/1291384117","state":["exists","up"]},{"osd":10,"uuid":"f3f31571-16de-4ef9-a951-f607776400a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6825","nonce":3121812139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6827","nonce":3121812139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6831","nonce":3121812139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3121812139},{"type":"v1","addr":"172.21.15.136:6829","nonce":3121812139}]},"public_addr":"172.21.15.136:6825/3121812139","cluster_addr":"172.21.15.136:6827/3121812139","heartbeat_back_addr":"172.21.15.136:6831/3121812139","heartbeat_front_addr":"172.21.15.136:6829/3121812139","state":["exists","up"]},{"osd":11,"uuid":"84a838c3-7bec-402f-a65a-78030b0380d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6825","nonce":2746302927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6827","nonce":2746302927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6831","nonce":2746302927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2746302927},{"type":"v1","addr":"172.21.15.172:6829","nonce":2746302927}]},"public_addr":"172.21.15.172:6825/2746302927","cluster_addr":"172.21.15.172:6827/2746302927","heartbeat_back_addr":"172.21.15.172:6831/2746302927","heartbeat_front_addr":"172.21.15.172:6829/2746302927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:57:55.728360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:20.946158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:58:46.472345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:15.960155+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:59:40.421580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:04.777339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:34.071189+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:00:58.793677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:24.364464+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:01:52.747477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:16.881059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T02:02:41.818250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":10,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":7,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/1646778488":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1801195624":"2024-09-17T01:56:28.359910+0000","172.21.15.103:0/1537689916":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6801/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:6801/623307529":"2024-09-17T01:56:28.359910+0000","172.21.15.103:6800/1247798496":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/2568354020":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/1667549073":"2024-09-17T01:56:01.936729+0000","172.21.15.103:0/3586311435":"2024-09-17T01:56:01.936729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T02:08:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 -- 172.21.15.103:0/57871141 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa9b80a32c0 msgr2=0x7fa9b80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa9b80a32c0 0x7fa9b80a5780 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009bd0 tx=0x7fa9c4005f90 comp rx=0 tx=0).stop 2024-09-16T02:08:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 -- 172.21.15.103:0/57871141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9d41049a0 msgr2=0x7fa9d419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9d41049a0 0x7fa9d419ab00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00d950 tx=0x7fa9bc00de10 comp rx=0 tx=0).stop 2024-09-16T02:08:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 -- 172.21.15.103:0/57871141 shutdown_connections 2024-09-16T02:08:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa9b80a32c0 0x7fa9b80a5780 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9d4109120 0x7fa9d419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9d41049a0 0x7fa9d419ab00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 --2- 172.21.15.103:0/57871141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d4103ff0 0x7fa9d419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 -- 172.21.15.103:0/57871141 >> 172.21.15.103:0/57871141 conn(0x7fa9d40ffa90 msgr2=0x7fa9d41004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 -- 172.21.15.103:0/57871141 shutdown_connections 2024-09-16T02:08:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:23.287+0000 7fa9da200640 1 -- 172.21.15.103:0/57871141 wait complete. 2024-09-16T02:08:23.903 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:23 smithi103 ceph-mon[30398]: pgmap v428: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T02:08:23.903 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:23 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/57871141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:08:23.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-16T02:08:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:23 smithi136 ceph-mon[35323]: pgmap v428: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T02:08:24.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:23 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/57871141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:08:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:23 smithi172 ceph-mon[35340]: pgmap v428: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T02:08:24.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:23 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/57871141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T02:08:24.312 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:08:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.851+0000 7fe513800640 1 -- 172.21.15.103:0/1204409351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe50c109120 msgr2=0x7fe50c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.851+0000 7fe513800640 1 --2- 172.21.15.103:0/1204409351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe50c109120 0x7fe50c110ca0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe504009ca0 tx=0x7fe50402a8c0 comp rx=0 tx=0).stop 2024-09-16T02:08:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.851+0000 7fe513800640 1 -- 172.21.15.103:0/1204409351 shutdown_connections 2024-09-16T02:08:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.851+0000 7fe513800640 1 --2- 172.21.15.103:0/1204409351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe50c109120 0x7fe50c110ca0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.851+0000 7fe513800640 1 --2- 172.21.15.103:0/1204409351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50c1049a0 0x7fe50c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.851+0000 7fe513800640 1 --2- 172.21.15.103:0/1204409351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50c103ff0 0x7fe50c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.851+0000 7fe513800640 1 -- 172.21.15.103:0/1204409351 >> 172.21.15.103:0/1204409351 conn(0x7fe50c0ffa90 msgr2=0x7fe50c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.852+0000 7fe513800640 1 -- 172.21.15.103:0/1204409351 shutdown_connections 2024-09-16T02:08:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.852+0000 7fe513800640 1 -- 172.21.15.103:0/1204409351 wait complete. 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe513800640 1 Processor -- start 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe513800640 1 -- start start 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe513800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe50c103ff0 0x7fe50c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe513800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50c1049a0 0x7fe50c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe513800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50c109120 0x7fe50c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe513800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1141a0 con 0x7fe50c103ff0 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe513800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c114020 con 0x7fe50c109120 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe513800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c114320 con 0x7fe50c1049a0 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe510e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe50c103ff0 0x7fe50c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe510e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe50c103ff0 0x7fe50c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58044/0 (socket says 172.21.15.103:58044) 2024-09-16T02:08:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.853+0000 7fe510e00640 1 -- 172.21.15.103:0/3066238599 learned_addr learned my addr 172.21.15.103:0/3066238599 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:08:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.854+0000 7fe503e00640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50c1049a0 0x7fe50c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.854+0000 7fe511800640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50c109120 0x7fe50c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.854+0000 7fe510e00640 1 -- 172.21.15.103:0/3066238599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50c1049a0 msgr2=0x7fe50c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.854+0000 7fe510e00640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50c1049a0 0x7fe50c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.854+0000 7fe510e00640 1 -- 172.21.15.103:0/3066238599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50c109120 msgr2=0x7fe50c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.854+0000 7fe510e00640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50c109120 0x7fe50c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.854+0000 7fe510e00640 1 -- 172.21.15.103:0/3066238599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c19f5b0 con 0x7fe50c103ff0 2024-09-16T02:08:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.854+0000 7fe510e00640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe50c103ff0 0x7fe50c19a600 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00e9f0 tx=0x7fe4fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.854+0000 7fe50a000640 1 -- 172.21.15.103:0/3066238599 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00cd80 con 0x7fe50c103ff0 2024-09-16T02:08:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.855+0000 7fe50a000640 1 -- 172.21.15.103:0/3066238599 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc005500 con 0x7fe50c103ff0 2024-09-16T02:08:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.855+0000 7fe513800640 1 -- 172.21.15.103:0/3066238599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c19f8a0 con 0x7fe50c103ff0 2024-09-16T02:08:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.855+0000 7fe50a000640 1 -- 172.21.15.103:0/3066238599 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc010690 con 0x7fe50c103ff0 2024-09-16T02:08:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.855+0000 7fe513800640 1 -- 172.21.15.103:0/3066238599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c1a7180 con 0x7fe50c103ff0 2024-09-16T02:08:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.856+0000 7fe513800640 1 -- 172.21.15.103:0/3066238599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50c1043d0 con 0x7fe50c103ff0 2024-09-16T02:08:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.858+0000 7fe50a000640 1 -- 172.21.15.103:0/3066238599 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe4fc0046c0 con 0x7fe50c103ff0 2024-09-16T02:08:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.859+0000 7fe50a000640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe4f00a32c0 0x7fe4f00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.859+0000 7fe50a000640 1 -- 172.21.15.103:0/3066238599 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8364+0+0 (secure 0 0 0) 0x7fe4fc0cfe20 con 0x7fe50c103ff0 2024-09-16T02:08:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.859+0000 7fe503e00640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe4f00a32c0 0x7fe4f00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.860+0000 7fe503e00640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe4f00a32c0 0x7fe4f00a5780 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe50c10ab20 tx=0x7fe4f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:24.863+0000 7fe50a000640 1 -- 172.21.15.103:0/3066238599 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4fc09ca60 con 0x7fe50c103ff0 2024-09-16T02:08:25.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.081+0000 7fe513800640 1 -- 172.21.15.103:0/3066238599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe50c19b960 con 0x7fe50c103ff0 2024-09-16T02:08:25.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.084+0000 7fe50a000640 1 -- 172.21.15.103:0/3066238599 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7fe4fc09c360 con 0x7fe50c103ff0 2024-09-16T02:08:25.085 INFO:teuthology.orchestra.run.smithi103.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T02:08:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.087+0000 7fe513800640 1 -- 172.21.15.103:0/3066238599 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe4f00a32c0 msgr2=0x7fe4f00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.087+0000 7fe513800640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe4f00a32c0 0x7fe4f00a5780 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe50c10ab20 tx=0x7fe4f4005f90 comp rx=0 tx=0).stop 2024-09-16T02:08:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.087+0000 7fe513800640 1 -- 172.21.15.103:0/3066238599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe50c103ff0 msgr2=0x7fe50c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.087+0000 7fe513800640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe50c103ff0 0x7fe50c19a600 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00e9f0 tx=0x7fe4fc00edf0 comp rx=0 tx=0).stop 2024-09-16T02:08:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.087+0000 7fe513800640 1 -- 172.21.15.103:0/3066238599 shutdown_connections 2024-09-16T02:08:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.087+0000 7fe513800640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fe4f00a32c0 0x7fe4f00a5780 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.087+0000 7fe513800640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50c109120 0x7fe50c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.087+0000 7fe513800640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50c1049a0 0x7fe50c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.087+0000 7fe513800640 1 --2- 172.21.15.103:0/3066238599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe50c103ff0 0x7fe50c19a600 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.087+0000 7fe513800640 1 -- 172.21.15.103:0/3066238599 >> 172.21.15.103:0/3066238599 conn(0x7fe50c0ffa90 msgr2=0x7fe50c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:25.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.088+0000 7fe513800640 1 -- 172.21.15.103:0/3066238599 shutdown_connections 2024-09-16T02:08:25.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:25.088+0000 7fe513800640 1 -- 172.21.15.103:0/3066238599 wait complete. 2024-09-16T02:08:25.408 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:25 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a[30394]: 2024-09-16T02:08:25.082+0000 7fa7cb000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T02:08:25.523 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-b[66066]: ignoring --setuser ceph since I am not root 2024-09-16T02:08:25.523 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-b[66066]: ignoring --setgroup ceph since I am not root 2024-09-16T02:08:25.523 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-mds[66070]: -- 172.21.15.136:0/3793291658 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56068603b860 con 0x560686019400 2024-09-16T02:08:25.523 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-b[66066]: starting mds.b at 2024-09-16T02:08:25.524 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e[65457]: ignoring --setuser ceph since I am not root 2024-09-16T02:08:25.524 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e[65457]: ignoring --setgroup ceph since I am not root 2024-09-16T02:08:25.524 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-mds[65461]: -- 172.21.15.136:0/2658730134 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562dcd733860 con 0x562dcd711000 2024-09-16T02:08:25.524 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e[65457]: starting mds.e at 2024-09-16T02:08:25.524 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h[65765]: ignoring --setuser ceph since I am not root 2024-09-16T02:08:25.524 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h[65765]: ignoring --setgroup ceph since I am not root 2024-09-16T02:08:25.524 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-mds[65769]: -- 172.21.15.136:0/496527094 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bda91a9860 con 0x55bda9186800 2024-09-16T02:08:25.525 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h[65765]: starting mds.h at 2024-09-16T02:08:25.545 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f[65536]: ignoring --setuser ceph since I am not root 2024-09-16T02:08:25.545 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f[65536]: ignoring --setgroup ceph since I am not root 2024-09-16T02:08:25.545 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-mds[65541]: -- 172.21.15.172:0/2495590520 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d800f3b860 con 0x55d800f19000 2024-09-16T02:08:25.545 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-16T02:08:25.545 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i[66149]: ignoring --setuser ceph since I am not root 2024-09-16T02:08:25.545 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i[66149]: ignoring --setgroup ceph since I am not root 2024-09-16T02:08:25.546 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-mds[66153]: -- 172.21.15.172:0/2731703769 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d789d3dc20 con 0x55d789d1b000 2024-09-16T02:08:25.546 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i[66149]: starting mds.i at 2024-09-16T02:08:25.546 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c[65852]: ignoring --setuser ceph since I am not root 2024-09-16T02:08:25.546 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c[65852]: ignoring --setgroup ceph since I am not root 2024-09-16T02:08:25.546 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-mds[65856]: -- 172.21.15.172:0/1394461922 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561bb0381860 con 0x561bb035f400 2024-09-16T02:08:25.546 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c[65852]: starting mds.c at 2024-09-16T02:08:25.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T02:08:26.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:25 smithi103 ceph-mon[30398]: pgmap v429: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T02:08:26.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:25 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3066238599' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T02:08:26.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:25 smithi103 ceph-mon[30398]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T02:08:26.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:25 smithi103 ceph-mon[30398]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T02:08:26.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:25 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/3066238599' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T02:08:26.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:25 smithi103 ceph-mon[30398]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T02:08:26.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:25 smithi103 ceph-mon[30398]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-16T02:08:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-mon[35323]: pgmap v429: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T02:08:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3066238599' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T02:08:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-mon[35323]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T02:08:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-mon[35323]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T02:08:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/3066238599' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T02:08:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-mon[35323]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T02:08:26.023 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:25 smithi136 ceph-mon[35323]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-16T02:08:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-mon[35340]: pgmap v429: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T02:08:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3066238599' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T02:08:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-mon[35340]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T02:08:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-mon[35340]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T02:08:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/3066238599' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T02:08:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-mon[35340]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T02:08:26.045 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:25 smithi172 ceph-mon[35340]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-16T02:08:26.221 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:08:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.758+0000 7fa386200640 1 -- 172.21.15.103:0/3334631636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa38010e3c0 msgr2=0x7fa380110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.758+0000 7fa386200640 1 --2- 172.21.15.103:0/3334631636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa38010e3c0 0x7fa380110810 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa378009ca0 tx=0x7fa37802a950 comp rx=0 tx=0).stop 2024-09-16T02:08:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.758+0000 7fa386200640 1 -- 172.21.15.103:0/3334631636 shutdown_connections 2024-09-16T02:08:26.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.758+0000 7fa386200640 1 --2- 172.21.15.103:0/3334631636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa38010e3c0 0x7fa380110810 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:26.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.758+0000 7fa386200640 1 --2- 172.21.15.103:0/3334631636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa380074f40 0x7fa38010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:26.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.758+0000 7fa386200640 1 --2- 172.21.15.103:0/3334631636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380074620 0x7fa380074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:26.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.758+0000 7fa386200640 1 -- 172.21.15.103:0/3334631636 >> 172.21.15.103:0/3334631636 conn(0x7fa3800fd8c0 msgr2=0x7fa3800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:26.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.758+0000 7fa386200640 1 -- 172.21.15.103:0/3334631636 shutdown_connections 2024-09-16T02:08:26.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.758+0000 7fa386200640 1 -- 172.21.15.103:0/3334631636 wait complete. 2024-09-16T02:08:26.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.759+0000 7fa386200640 1 Processor -- start 2024-09-16T02:08:26.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.759+0000 7fa386200640 1 -- start start 2024-09-16T02:08:26.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.759+0000 7fa386200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380074620 0x7fa38019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:26.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.759+0000 7fa386200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380074f40 0x7fa38019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:26.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.759+0000 7fa386200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa38010e3c0 0x7fa38019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.759+0000 7fa386200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801141a0 con 0x7fa380074f40 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.759+0000 7fa386200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa380114020 con 0x7fa38010e3c0 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.759+0000 7fa386200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa380114320 con 0x7fa380074620 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa37f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380074620 0x7fa38019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa376a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380074f40 0x7fa38019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa37f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380074620 0x7fa38019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.103:51092/0 (socket says 172.21.15.103:51092) 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa37f400640 1 -- 172.21.15.103:0/976492601 learned_addr learned my addr 172.21.15.103:0/976492601 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa37fe00640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa38010e3c0 0x7fa38019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa376a00640 1 -- 172.21.15.103:0/976492601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380074620 msgr2=0x7fa38019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa376a00640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380074620 0x7fa38019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa376a00640 1 -- 172.21.15.103:0/976492601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa38010e3c0 msgr2=0x7fa38019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa376a00640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa38010e3c0 0x7fa38019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa376a00640 1 -- 172.21.15.103:0/976492601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38019f5f0 con 0x7fa380074f40 2024-09-16T02:08:26.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.760+0000 7fa376a00640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380074f40 0x7fa38019ab80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa368005fd0 tx=0x7fa36800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:26.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.761+0000 7fa37cc00640 1 -- 172.21.15.103:0/976492601 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368009c30 con 0x7fa380074f40 2024-09-16T02:08:26.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.761+0000 7fa37cc00640 1 -- 172.21.15.103:0/976492601 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa368009dd0 con 0x7fa380074f40 2024-09-16T02:08:26.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.761+0000 7fa386200640 1 -- 172.21.15.103:0/976492601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38019f8e0 con 0x7fa380074f40 2024-09-16T02:08:26.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.761+0000 7fa37cc00640 1 -- 172.21.15.103:0/976492601 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368010460 con 0x7fa380074f40 2024-09-16T02:08:26.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.761+0000 7fa386200640 1 -- 172.21.15.103:0/976492601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3801a71c0 con 0x7fa380074f40 2024-09-16T02:08:26.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.762+0000 7fa386200640 1 -- 172.21.15.103:0/976492601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa380074a00 con 0x7fa380074f40 2024-09-16T02:08:26.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.764+0000 7fa37cc00640 1 -- 172.21.15.103:0/976492601 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa368003f50 con 0x7fa380074f40 2024-09-16T02:08:26.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.765+0000 7fa37cc00640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa3640a3240 0x7fa3640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:26.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.766+0000 7fa37cc00640 1 -- 172.21.15.103:0/976492601 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8622+0+0 (secure 0 0 0) 0x7fa3680cf660 con 0x7fa380074f40 2024-09-16T02:08:26.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.766+0000 7fa37f400640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa3640a3240 0x7fa3640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:26.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.767+0000 7fa37f400640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa3640a3240 0x7fa3640a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa370005fd0 tx=0x7fa37000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:26.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.770+0000 7fa37cc00640 1 -- 172.21.15.103:0/976492601 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa368017020 con 0x7fa380074f40 2024-09-16T02:08:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:26.978+0000 7fa386200640 1 -- 172.21.15.103:0/976492601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa380108fb0 con 0x7fa380074f40 2024-09-16T02:08:27.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.092+0000 7fa37cc00640 1 -- 172.21.15.103:0/976492601 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fa36809bb50 con 0x7fa380074f40 2024-09-16T02:08:27.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.094+0000 7fa386200640 1 -- 172.21.15.103:0/976492601 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa3640a3240 msgr2=0x7fa3640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:27.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.094+0000 7fa386200640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa3640a3240 0x7fa3640a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa370005fd0 tx=0x7fa37000b4a0 comp rx=0 tx=0).stop 2024-09-16T02:08:27.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.094+0000 7fa386200640 1 -- 172.21.15.103:0/976492601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380074f40 msgr2=0x7fa38019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:27.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.094+0000 7fa386200640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380074f40 0x7fa38019ab80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa368005fd0 tx=0x7fa36800dde0 comp rx=0 tx=0).stop 2024-09-16T02:08:27.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.095+0000 7fa386200640 1 -- 172.21.15.103:0/976492601 shutdown_connections 2024-09-16T02:08:27.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.095+0000 7fa386200640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7fa3640a3240 0x7fa3640a5700 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:27.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.095+0000 7fa386200640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa38010e3c0 0x7fa38019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:27.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.095+0000 7fa386200640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380074f40 0x7fa38019ab80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:27.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.095+0000 7fa386200640 1 --2- 172.21.15.103:0/976492601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380074620 0x7fa38019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:27.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.095+0000 7fa386200640 1 -- 172.21.15.103:0/976492601 >> 172.21.15.103:0/976492601 conn(0x7fa3800fd8c0 msgr2=0x7fa3800fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:27.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.095+0000 7fa386200640 1 -- 172.21.15.103:0/976492601 shutdown_connections 2024-09-16T02:08:27.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:27.095+0000 7fa386200640 1 -- 172.21.15.103:0/976492601 wait complete. 2024-09-16T02:08:27.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.136:6834/1574073951,v1:172.21.15.136:6836/1574073951] up:boot 2024-09-16T02:08:27.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.136:6835/2665091705,v1:172.21.15.136:6838/2665091705] up:boot 2024-09-16T02:08:27.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.172:6834/2803458455,v1:172.21.15.172:6835/2803458455] up:boot 2024-09-16T02:08:27.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.136:6837/3142734959,v1:172.21.15.136:6839/3142734959] up:boot 2024-09-16T02:08:27.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.172:6836/854351850,v1:172.21.15.172:6837/854351850] up:boot 2024-09-16T02:08:27.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: mds.? [v2:172.21.15.172:6838/4210310064,v1:172.21.15.172:6839/4210310064] up:boot 2024-09-16T02:08:27.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-16T02:08:27.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:08:27.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:08:27.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:08:27.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:08:27.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:08:27.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:08:27.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: pgmap v431: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T02:08:27.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:27 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/976492601' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T02:08:27.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.136:6834/1574073951,v1:172.21.15.136:6836/1574073951] up:boot 2024-09-16T02:08:27.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.136:6835/2665091705,v1:172.21.15.136:6838/2665091705] up:boot 2024-09-16T02:08:27.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.172:6834/2803458455,v1:172.21.15.172:6835/2803458455] up:boot 2024-09-16T02:08:27.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.136:6837/3142734959,v1:172.21.15.136:6839/3142734959] up:boot 2024-09-16T02:08:27.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.172:6836/854351850,v1:172.21.15.172:6837/854351850] up:boot 2024-09-16T02:08:27.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: mds.? [v2:172.21.15.172:6838/4210310064,v1:172.21.15.172:6839/4210310064] up:boot 2024-09-16T02:08:27.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-16T02:08:27.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:08:27.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:08:27.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:08:27.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:08:27.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:08:27.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:08:27.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: pgmap v431: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T02:08:27.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:27 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/976492601' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T02:08:27.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.136:6834/1574073951,v1:172.21.15.136:6836/1574073951] up:boot 2024-09-16T02:08:27.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.136:6835/2665091705,v1:172.21.15.136:6838/2665091705] up:boot 2024-09-16T02:08:27.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.172:6834/2803458455,v1:172.21.15.172:6835/2803458455] up:boot 2024-09-16T02:08:27.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.136:6837/3142734959,v1:172.21.15.136:6839/3142734959] up:boot 2024-09-16T02:08:27.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.172:6836/854351850,v1:172.21.15.172:6837/854351850] up:boot 2024-09-16T02:08:27.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: mds.? [v2:172.21.15.172:6838/4210310064,v1:172.21.15.172:6839/4210310064] up:boot 2024-09-16T02:08:27.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-16T02:08:27.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T02:08:27.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T02:08:27.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T02:08:27.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T02:08:27.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T02:08:27.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: from='mgr.14150 172.21.15.103:0/3818753392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T02:08:27.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: pgmap v431: 97 pgs: 97 active+clean; 8.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-16T02:08:27.546 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:27 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/976492601' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T02:08:27.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T02:08:28.137 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:08:28.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:28 smithi103 ceph-mon[30398]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T02:08:28.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:28 smithi103 ceph-mon[30398]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T02:08:28.410 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:28 smithi103 ceph-mon[30398]: Cluster is now healthy 2024-09-16T02:08:28.410 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:28 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/976492601' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T02:08:28.410 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:28 smithi103 ceph-mon[30398]: fsmap 12 up:standby 2024-09-16T02:08:28.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:28 smithi136 ceph-mon[35323]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T02:08:28.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:28 smithi136 ceph-mon[35323]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T02:08:28.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:28 smithi136 ceph-mon[35323]: Cluster is now healthy 2024-09-16T02:08:28.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:28 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/976492601' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T02:08:28.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:28 smithi136 ceph-mon[35323]: fsmap 12 up:standby 2024-09-16T02:08:28.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:28 smithi172 ceph-mon[35340]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T02:08:28.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:28 smithi172 ceph-mon[35340]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T02:08:28.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:28 smithi172 ceph-mon[35340]: Cluster is now healthy 2024-09-16T02:08:28.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:28 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/976492601' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T02:08:28.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:28 smithi172 ceph-mon[35340]: fsmap 12 up:standby 2024-09-16T02:08:28.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.671+0000 7f1d83c00640 1 -- 172.21.15.103:0/582558187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d7c102790 msgr2=0x7f1d7c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:28.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.671+0000 7f1d83c00640 1 --2- 172.21.15.103:0/582558187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d7c102790 0x7f1d7c1067e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009900 tx=0x7f1d6402a3f0 comp rx=0 tx=0).stop 2024-09-16T02:08:28.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.671+0000 7f1d83c00640 1 -- 172.21.15.103:0/582558187 shutdown_connections 2024-09-16T02:08:28.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.671+0000 7f1d83c00640 1 --2- 172.21.15.103:0/582558187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c106f10 0x7f1d7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:28.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.671+0000 7f1d83c00640 1 --2- 172.21.15.103:0/582558187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d7c102790 0x7f1d7c1067e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:28.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.671+0000 7f1d83c00640 1 --2- 172.21.15.103:0/582558187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d7c101de0 0x7f1d7c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:28.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.671+0000 7f1d83c00640 1 -- 172.21.15.103:0/582558187 >> 172.21.15.103:0/582558187 conn(0x7f1d7c0fd8c0 msgr2=0x7f1d7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:28.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.671+0000 7f1d83c00640 1 -- 172.21.15.103:0/582558187 shutdown_connections 2024-09-16T02:08:28.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.671+0000 7f1d83c00640 1 -- 172.21.15.103:0/582558187 wait complete. 2024-09-16T02:08:28.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.672+0000 7f1d83c00640 1 Processor -- start 2024-09-16T02:08:28.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.672+0000 7f1d83c00640 1 -- start start 2024-09-16T02:08:28.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.672+0000 7f1d83c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c101de0 0x7f1d7c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:28.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.672+0000 7f1d83c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d7c102790 0x7f1d7c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.672+0000 7f1d83c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d7c106f10 0x7f1d7c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.672+0000 7f1d83c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c111f70 con 0x7f1d7c106f10 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.672+0000 7f1d83c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c111df0 con 0x7f1d7c102790 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.672+0000 7f1d83c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c1120f0 con 0x7f1d7c101de0 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d81c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d7c106f10 0x7f1d7c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d73e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d7c102790 0x7f1d7c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d73e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d7c102790 0x7f1d7c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.103:55260/0 (socket says 172.21.15.103:55260) 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d81200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c101de0 0x7f1d7c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d73e00640 1 -- 172.21.15.103:0/588047335 learned_addr learned my addr 172.21.15.103:0/588047335 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d73e00640 1 -- 172.21.15.103:0/588047335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c101de0 msgr2=0x7f1d7c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d73e00640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c101de0 0x7f1d7c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:28.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d73e00640 1 -- 172.21.15.103:0/588047335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d7c106f10 msgr2=0x7f1d7c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:28.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d73e00640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d7c106f10 0x7f1d7c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:28.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d73e00640 1 -- 172.21.15.103:0/588047335 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7c1a68e0 con 0x7f1d7c102790 2024-09-16T02:08:28.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.673+0000 7f1d73e00640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d7c102790 0x7f1d7c074a30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d64006010 tx=0x7f1d64004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:28.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.674+0000 7f1d7a000640 1 -- 172.21.15.103:0/588047335 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6403f030 con 0x7f1d7c102790 2024-09-16T02:08:28.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.674+0000 7f1d83c00640 1 -- 172.21.15.103:0/588047335 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7c1a6b70 con 0x7f1d7c102790 2024-09-16T02:08:28.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.674+0000 7f1d7a000640 1 -- 172.21.15.103:0/588047335 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d64004440 con 0x7f1d7c102790 2024-09-16T02:08:28.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.674+0000 7f1d7a000640 1 -- 172.21.15.103:0/588047335 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64035070 con 0x7f1d7c102790 2024-09-16T02:08:28.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.674+0000 7f1d83c00640 1 -- 172.21.15.103:0/588047335 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7c1a70a0 con 0x7f1d7c102790 2024-09-16T02:08:28.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.675+0000 7f1d83c00640 1 -- 172.21.15.103:0/588047335 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d7c1021c0 con 0x7f1d7c102790 2024-09-16T02:08:28.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.677+0000 7f1d7a000640 1 -- 172.21.15.103:0/588047335 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1d64042030 con 0x7f1d7c102790 2024-09-16T02:08:28.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.678+0000 7f1d7a000640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1d600a3240 0x7f1d600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:28.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.679+0000 7f1d7a000640 1 -- 172.21.15.103:0/588047335 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8622+0+0 (secure 0 0 0) 0x7f1d640f00a0 con 0x7f1d7c102790 2024-09-16T02:08:28.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.679+0000 7f1d81200640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1d600a3240 0x7f1d600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:28.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.679+0000 7f1d81200640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1d600a3240 0x7f1d600a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0095f0 tx=0x7f1d6c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:28.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.681+0000 7f1d7a000640 1 -- 172.21.15.103:0/588047335 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d640bc590 con 0x7f1d7c102790 2024-09-16T02:08:28.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:28.856+0000 7f1d83c00640 1 -- 172.21.15.103:0/588047335 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1d7c10a150 con 0x7f1d7c102790 2024-09-16T02:08:29.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.105+0000 7f1d7a000640 1 -- 172.21.15.103:0/588047335 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) ==== 181+0+0 (secure 0 0 0) 0x7f1d640c0890 con 0x7f1d7c102790 2024-09-16T02:08:29.106 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' removed 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.107+0000 7f1d83c00640 1 -- 172.21.15.103:0/588047335 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1d600a3240 msgr2=0x7f1d600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.107+0000 7f1d83c00640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1d600a3240 0x7f1d600a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0095f0 tx=0x7f1d6c005f90 comp rx=0 tx=0).stop 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.107+0000 7f1d83c00640 1 -- 172.21.15.103:0/588047335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d7c102790 msgr2=0x7f1d7c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.107+0000 7f1d83c00640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d7c102790 0x7f1d7c074a30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d64006010 tx=0x7f1d64004040 comp rx=0 tx=0).stop 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.107+0000 7f1d83c00640 1 -- 172.21.15.103:0/588047335 shutdown_connections 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.107+0000 7f1d83c00640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f1d600a3240 0x7f1d600a5700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.107+0000 7f1d83c00640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d7c106f10 0x7f1d7c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.107+0000 7f1d83c00640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d7c102790 0x7f1d7c074a30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.107+0000 7f1d83c00640 1 --2- 172.21.15.103:0/588047335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c101de0 0x7f1d7c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.107+0000 7f1d83c00640 1 -- 172.21.15.103:0/588047335 >> 172.21.15.103:0/588047335 conn(0x7f1d7c0fd8c0 msgr2=0x7f1d7c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.108+0000 7f1d83c00640 1 -- 172.21.15.103:0/588047335 shutdown_connections 2024-09-16T02:08:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:29.108+0000 7f1d83c00640 1 -- 172.21.15.103:0/588047335 wait complete. 2024-09-16T02:08:29.435 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:29 smithi103 ceph-mon[30398]: pgmap v432: 97 pgs: 97 active+clean; 8.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 409 B/s wr, 5 op/s 2024-09-16T02:08:29.436 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:29 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/588047335' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:08:29.436 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:29 smithi103 ceph-mon[30398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:08:29.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:29 smithi136 ceph-mon[35323]: pgmap v432: 97 pgs: 97 active+clean; 8.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 409 B/s wr, 5 op/s 2024-09-16T02:08:29.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:29 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/588047335' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:08:29.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:29 smithi136 ceph-mon[35323]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:08:29.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:29 smithi172 ceph-mon[35340]: pgmap v432: 97 pgs: 97 active+clean; 8.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 409 B/s wr, 5 op/s 2024-09-16T02:08:29.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:29 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/588047335' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:08:29.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:29 smithi172 ceph-mon[35340]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:08:29.916 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T02:08:30.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:30 smithi103 ceph-mon[30398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:08:30.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:30 smithi103 ceph-mon[30398]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T02:08:30.289 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:08:30.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:30 smithi136 ceph-mon[35323]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:08:30.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:30 smithi136 ceph-mon[35323]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T02:08:30.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:30 smithi172 ceph-mon[35340]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:08:30.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:30 smithi172 ceph-mon[35340]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T02:08:30.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.826+0000 7f77e5e00640 1 -- 172.21.15.103:0/3746355527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e01049c0 msgr2=0x7f77e0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:30.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.826+0000 7f77e5e00640 1 --2- 172.21.15.103:0/3746355527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e01049c0 0x7f77e0108a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009900 tx=0x7f77c802a3f0 comp rx=0 tx=0).stop 2024-09-16T02:08:30.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.827+0000 7f77e5e00640 1 -- 172.21.15.103:0/3746355527 shutdown_connections 2024-09-16T02:08:30.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.827+0000 7f77e5e00640 1 --2- 172.21.15.103:0/3746355527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77e0109140 0x7f77e0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:30.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.827+0000 7f77e5e00640 1 --2- 172.21.15.103:0/3746355527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e01049c0 0x7f77e0108a10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:30.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.827+0000 7f77e5e00640 1 --2- 172.21.15.103:0/3746355527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e0104010 0x7f77e01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:30.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.827+0000 7f77e5e00640 1 -- 172.21.15.103:0/3746355527 >> 172.21.15.103:0/3746355527 conn(0x7f77e00ffa50 msgr2=0x7f77e0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:30.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.827+0000 7f77e5e00640 1 -- 172.21.15.103:0/3746355527 shutdown_connections 2024-09-16T02:08:30.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.827+0000 7f77e5e00640 1 -- 172.21.15.103:0/3746355527 wait complete. 2024-09-16T02:08:30.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77e5e00640 1 Processor -- start 2024-09-16T02:08:30.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77e5e00640 1 -- start start 2024-09-16T02:08:30.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77e5e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e0104010 0x7f77e019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:30.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77e5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e01049c0 0x7f77e019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:30.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77e5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77e0109140 0x7f77e019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:30.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77e5e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77e0114140 con 0x7f77e0104010 2024-09-16T02:08:30.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77e5e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77e0113fc0 con 0x7f77e0109140 2024-09-16T02:08:30.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77e5e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f77e01142c0 con 0x7f77e01049c0 2024-09-16T02:08:30.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77dea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e0104010 0x7f77e019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:30.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77dea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e0104010 0x7f77e019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56850/0 (socket says 172.21.15.103:56850) 2024-09-16T02:08:30.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.828+0000 7f77dea00640 1 -- 172.21.15.103:0/4227782497 learned_addr learned my addr 172.21.15.103:0/4227782497 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-16T02:08:30.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.829+0000 7f77d7e00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e01049c0 0x7f77e019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:30.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.829+0000 7f77df400640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77e0109140 0x7f77e019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:30.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.829+0000 7f77dea00640 1 -- 172.21.15.103:0/4227782497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e01049c0 msgr2=0x7f77e019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:30.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.829+0000 7f77dea00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e01049c0 0x7f77e019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:30.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.829+0000 7f77dea00640 1 -- 172.21.15.103:0/4227782497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77e0109140 msgr2=0x7f77e019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:30.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.829+0000 7f77dea00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77e0109140 0x7f77e019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:30.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.829+0000 7f77dea00640 1 -- 172.21.15.103:0/4227782497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e019f610 con 0x7f77e0104010 2024-09-16T02:08:30.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.829+0000 7f77dea00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e0104010 0x7f77e019a660 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f77d000d950 tx=0x7f77d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:30.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.830+0000 7f77e4a00640 1 -- 172.21.15.103:0/4227782497 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0017070 con 0x7f77e0104010 2024-09-16T02:08:30.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.830+0000 7f77e4a00640 1 -- 172.21.15.103:0/4227782497 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0009c30 con 0x7f77e0104010 2024-09-16T02:08:30.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.830+0000 7f77e5e00640 1 -- 172.21.15.103:0/4227782497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e019f900 con 0x7f77e0104010 2024-09-16T02:08:30.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.830+0000 7f77e4a00640 1 -- 172.21.15.103:0/4227782497 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d000f6b0 con 0x7f77e0104010 2024-09-16T02:08:30.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.830+0000 7f77e5e00640 1 -- 172.21.15.103:0/4227782497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e01a71e0 con 0x7f77e0104010 2024-09-16T02:08:30.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.831+0000 7f77e5e00640 1 -- 172.21.15.103:0/4227782497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e01043f0 con 0x7f77e0104010 2024-09-16T02:08:30.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.833+0000 7f77e4a00640 1 -- 172.21.15.103:0/4227782497 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f77d00109d0 con 0x7f77e0104010 2024-09-16T02:08:30.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.834+0000 7f77e4a00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f77c40a3240 0x7f77c40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:08:30.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.834+0000 7f77e4a00640 1 -- 172.21.15.103:0/4227782497 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7973+0+0 (secure 0 0 0) 0x7f77d00cf3a0 con 0x7f77e0104010 2024-09-16T02:08:30.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.834+0000 7f77d7e00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f77c40a3240 0x7f77c40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:08:30.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.835+0000 7f77d7e00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f77c40a3240 0x7f77c40a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009230 tx=0x7f77c80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:08:30.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:30.838+0000 7f77e4a00640 1 -- 172.21.15.103:0/4227782497 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77d0009de0 con 0x7f77e0104010 2024-09-16T02:08:31.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.012+0000 7f77e5e00640 1 -- 172.21.15.103:0/4227782497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f77e010c3e0 con 0x7f77e0104010 2024-09-16T02:08:31.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.106+0000 7f77e4a00640 1 -- 172.21.15.103:0/4227782497 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) ==== 169+0+0 (secure 0 0 0) 0x7f77d009bbc0 con 0x7f77e0104010 2024-09-16T02:08:31.108 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' removed 2024-09-16T02:08:31.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 -- 172.21.15.103:0/4227782497 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f77c40a3240 msgr2=0x7f77c40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:31.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f77c40a3240 0x7f77c40a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009230 tx=0x7f77c80058d0 comp rx=0 tx=0).stop 2024-09-16T02:08:31.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 -- 172.21.15.103:0/4227782497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e0104010 msgr2=0x7f77e019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:08:31.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e0104010 0x7f77e019a660 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f77d000d950 tx=0x7f77d000de10 comp rx=0 tx=0).stop 2024-09-16T02:08:31.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 -- 172.21.15.103:0/4227782497 shutdown_connections 2024-09-16T02:08:31.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.103:6800/3075264030,v1:172.21.15.103:6801/3075264030] conn(0x7f77c40a3240 0x7f77c40a5700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:31.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77e0109140 0x7f77e019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:31.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e01049c0 0x7f77e019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:31.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 --2- 172.21.15.103:0/4227782497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e0104010 0x7f77e019a660 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:08:31.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 -- 172.21.15.103:0/4227782497 >> 172.21.15.103:0/4227782497 conn(0x7f77e00ffa50 msgr2=0x7f77e0100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:08:31.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 -- 172.21.15.103:0/4227782497 shutdown_connections 2024-09-16T02:08:31.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-16T02:08:31.110+0000 7f77e5e00640 1 -- 172.21.15.103:0/4227782497 wait complete. 2024-09-16T02:08:31.451 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:31 smithi103 ceph-mon[30398]: pgmap v434: 65 pgs: 65 active+clean; 577 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:08:31.451 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:31 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/4227782497' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:08:31.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:31 smithi136 ceph-mon[35323]: pgmap v434: 65 pgs: 65 active+clean; 577 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:08:31.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:31 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/4227782497' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:08:31.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:31 smithi172 ceph-mon[35340]: pgmap v434: 65 pgs: 65 active+clean; 577 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:08:31.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:31 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/4227782497' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:08:31.925 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:08:31.985 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:08:32.017 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:08:32.050 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T02:08:32.051 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T02:08:32.086 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T02:08:32.104 DEBUG:teuthology.orchestra.run.smithi172:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T02:08:32.121 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T02:08:32.122 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T02:08:32.122 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.a 2024-09-16T02:08:32.176 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:32 smithi103 ceph-mon[30398]: from='client.? 172.21.15.103:0/4227782497' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:08:32.176 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:32 smithi103 ceph-mon[30398]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T02:08:32.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:32 smithi136 ceph-mon[35323]: from='client.? 172.21.15.103:0/4227782497' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:08:32.273 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:32 smithi136 ceph-mon[35323]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T02:08:32.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:32 smithi103 systemd[1]: Stopping Ceph mon.a for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:32.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:32 smithi172 ceph-mon[35340]: from='client.? 172.21.15.103:0/4227782497' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:08:32.545 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:32 smithi172 ceph-mon[35340]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T02:08:32.982 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a[30394]: 2024-09-16T02:08:32.533+0000 7fa7d1e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:32.982 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a[30394]: 2024-09-16T02:08:32.533+0000 7fa7d1e00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T02:08:32.982 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:32 smithi103 podman[88548]: 2024-09-16 02:08:32.72843832 +0000 UTC m=+0.324529943 container died e60b64d004a06b198ed5ddbd90e8194edbffd30db141086ff1f9287511db2fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-16T02:08:33.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:33 smithi103 podman[88548]: 2024-09-16 02:08:33.197023079 +0000 UTC m=+0.793114701 container cleanup e60b64d004a06b198ed5ddbd90e8194edbffd30db141086ff1f9287511db2fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T02:08:33.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:33 smithi103 bash[88548]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a 2024-09-16T02:08:33.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 16 02:08:33 smithi103 podman[88561]: 2024-09-16 02:08:33.398335403 +0000 UTC m=+0.662828547 container remove e60b64d004a06b198ed5ddbd90e8194edbffd30db141086ff1f9287511db2fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T02:08:33.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T02:08:33.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:33.769 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T02:08:33.769 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-16T02:08:33.769 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.b 2024-09-16T02:08:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:33 smithi136 systemd[1]: Stopping Ceph mon.b for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:34.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:34 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-b[35319]: 2024-09-16T02:08:34.050+0000 7f557e685640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:34.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:34 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-b[35319]: 2024-09-16T02:08:34.050+0000 7f557e685640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T02:08:34.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:34 smithi136 podman[68737]: 2024-09-16 02:08:34.20437246 +0000 UTC m=+0.246348334 container died 66e9a9454b023ccd9af4643b53dbc95928df0f6c70e15eba56d8bb2798bf63d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-09-16T02:08:34.970 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:34 smithi136 podman[68737]: 2024-09-16 02:08:34.717537894 +0000 UTC m=+0.759513767 container cleanup 66e9a9454b023ccd9af4643b53dbc95928df0f6c70e15eba56d8bb2798bf63d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T02:08:34.970 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:34 smithi136 bash[68737]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-b 2024-09-16T02:08:35.214 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T02:08:35.236 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:34 smithi136 podman[68749]: 2024-09-16 02:08:34.969198909 +0000 UTC m=+0.760503027 container remove 66e9a9454b023ccd9af4643b53dbc95928df0f6c70e15eba56d8bb2798bf63d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T02:08:35.236 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:35 smithi136 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-16T02:08:35.237 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:35 smithi136 systemd[1]: Stopped Ceph mon.b for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:08:35.237 INFO:journalctl@ceph.mon.b.smithi136.stdout:Sep 16 02:08:35 smithi136 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.b.service: Consumed 8.401s CPU time. 2024-09-16T02:08:35.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:35.271 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-16T02:08:35.271 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-16T02:08:35.272 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.c 2024-09-16T02:08:35.586 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:35 smithi172 systemd[1]: Stopping Ceph mon.c for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:35.927 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:35 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-c[35336]: 2024-09-16T02:08:35.584+0000 7fd712203640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:35.927 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:35 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-c[35336]: 2024-09-16T02:08:35.584+0000 7fd712203640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T02:08:35.928 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:35 smithi172 podman[68946]: 2024-09-16 02:08:35.656290285 +0000 UTC m=+0.179214736 container died 9e5b8f07d99b33ea7b16072cc45426c2d39c19a7bd488b370303ada7e7a2bb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-16T02:08:36.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:36 smithi172 podman[68946]: 2024-09-16 02:08:36.043043269 +0000 UTC m=+0.565967717 container cleanup 9e5b8f07d99b33ea7b16072cc45426c2d39c19a7bd488b370303ada7e7a2bb20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:08:36.295 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 16 02:08:36 smithi172 bash[68946]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mon-c 2024-09-16T02:08:36.476 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T02:08:36.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:36.523 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-16T02:08:36.523 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T02:08:36.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.x 2024-09-16T02:08:36.934 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:08:36 smithi103 systemd[1]: Stopping Ceph mgr.x for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:37.254 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:08:36 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:08:36.932+0000 7fcecd600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:37.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:08:36 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x[30635]: 2024-09-16T02:08:36.932+0000 7fcecd600640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-16T02:08:37.255 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:08:37 smithi103 podman[88668]: 2024-09-16 02:08:37.014599218 +0000 UTC m=+0.207390245 container died 9e0b235d2a04d65679b8c80e0da3a7f5ccac8daaf062a2a99b95d6912c863652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T02:08:37.549 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:08:37 smithi103 podman[88668]: 2024-09-16 02:08:37.529341496 +0000 UTC m=+0.722132508 container cleanup 9e0b235d2a04d65679b8c80e0da3a7f5ccac8daaf062a2a99b95d6912c863652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:08:37.549 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:08:37 smithi103 bash[88668]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x 2024-09-16T02:08:38.046 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 16 02:08:37 smithi103 podman[88683]: 2024-09-16 02:08:37.793108965 +0000 UTC m=+0.768412416 container remove 9e0b235d2a04d65679b8c80e0da3a7f5ccac8daaf062a2a99b95d6912c863652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e) 2024-09-16T02:08:38.067 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T02:08:38.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:38.139 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T02:08:38.139 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T02:08:38.139 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.y 2024-09-16T02:08:38.461 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 02:08:38 smithi136 systemd[1]: Stopping Ceph mgr.y for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:38.772 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 02:08:38 smithi136 podman[68864]: 2024-09-16 02:08:38.460050929 +0000 UTC m=+0.134993050 container died 1060f40eb5055637d04398a3f8a650d0bb4ed94ecb42252a1184a6aa6ed3f190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:08:38.772 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 02:08:38 smithi136 podman[68864]: 2024-09-16 02:08:38.771603174 +0000 UTC m=+0.446545304 container cleanup 1060f40eb5055637d04398a3f8a650d0bb4ed94ecb42252a1184a6aa6ed3f190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-16T02:08:39.023 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 02:08:38 smithi136 bash[68864]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y 2024-09-16T02:08:39.023 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 02:08:38 smithi136 podman[68876]: 2024-09-16 02:08:38.939845123 +0000 UTC m=+0.474862566 container remove 1060f40eb5055637d04398a3f8a650d0bb4ed94ecb42252a1184a6aa6ed3f190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True) 2024-09-16T02:08:39.023 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Sep 16 02:08:38 smithi136 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-16T02:08:39.200 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T02:08:39.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:39.235 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T02:08:39.236 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-16T02:08:39.236 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.z 2024-09-16T02:08:39.545 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 02:08:39 smithi172 systemd[1]: Stopping Ceph mgr.z for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:39.942 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 02:08:39 smithi172 podman[69082]: 2024-09-16 02:08:39.661010934 +0000 UTC m=+0.234314573 container died bb33a43d08a82ea80709c7d2e291b26063bb06c26367d8efec0ef29aaa69b9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T02:08:40.284 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 02:08:40 smithi172 podman[69082]: 2024-09-16 02:08:40.031318643 +0000 UTC m=+0.604622271 container cleanup bb33a43d08a82ea80709c7d2e291b26063bb06c26367d8efec0ef29aaa69b9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-09-16T02:08:40.284 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 16 02:08:40 smithi172 bash[69082]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mgr-z 2024-09-16T02:08:40.406 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-16T02:08:40.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:40.449 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-16T02:08:40.449 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T02:08:40.449 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.0 2024-09-16T02:08:40.754 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:40 smithi103 systemd[1]: Stopping Ceph osd.0 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:41.255 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:40 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0[44237]: 2024-09-16T02:08:40.844+0000 7fb1f5400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:41.255 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:40 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0[44237]: 2024-09-16T02:08:40.844+0000 7fb1f5400640 -1 osd.0 84 *** Got signal Terminated *** 2024-09-16T02:08:41.255 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:40 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0[44237]: 2024-09-16T02:08:40.844+0000 7fb1f5400640 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:08:46.637 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:46 smithi103 podman[88788]: 2024-09-16 02:08:46.335092749 +0000 UTC m=+5.602177983 container died cab308f332a91ee7d408fab0c7f68826d2a5281f936bc3fd1d723c6e536ee925 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T02:08:47.005 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:46 smithi103 podman[88788]: 2024-09-16 02:08:46.826085154 +0000 UTC m=+6.093170386 container cleanup cab308f332a91ee7d408fab0c7f68826d2a5281f936bc3fd1d723c6e536ee925 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:08:47.005 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:46 smithi103 bash[88788]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0 2024-09-16T02:08:47.257 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:47 smithi103 podman[88806]: 2024-09-16 02:08:47.010569043 +0000 UTC m=+0.656363133 container remove cab308f332a91ee7d408fab0c7f68826d2a5281f936bc3fd1d723c6e536ee925 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T02:08:47.832 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:47 smithi103 podman[88889]: 2024-09-16 02:08:47.452911699 +0000 UTC m=+0.027842743 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:08:47.832 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:47 smithi103 podman[88889]: 2024-09-16 02:08:47.586945269 +0000 UTC m=+0.161876292 container create e0c5c2ee970453d8f3009abf65f060cfb5bf275d440181ae197aeebb069403d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T02:08:47.833 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:47 smithi103 podman[88889]: 2024-09-16 02:08:47.831176694 +0000 UTC m=+0.406107745 container init e0c5c2ee970453d8f3009abf65f060cfb5bf275d440181ae197aeebb069403d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T02:08:48.098 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:47 smithi103 podman[88889]: 2024-09-16 02:08:47.838200058 +0000 UTC m=+0.413131087 container start e0c5c2ee970453d8f3009abf65f060cfb5bf275d440181ae197aeebb069403d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:08:48.098 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:47 smithi103 podman[88889]: 2024-09-16 02:08:47.914357129 +0000 UTC m=+0.489288165 container attach e0c5c2ee970453d8f3009abf65f060cfb5bf275d440181ae197aeebb069403d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:08:48.427 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:48 smithi103 podman[88889]: 2024-09-16 02:08:48.097688791 +0000 UTC m=+0.672619834 container died e0c5c2ee970453d8f3009abf65f060cfb5bf275d440181ae197aeebb069403d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:08:48.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T02:08:49.004 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:48 smithi103 podman[88911]: 2024-09-16 02:08:48.901886576 +0000 UTC m=+0.790768706 container remove e0c5c2ee970453d8f3009abf65f060cfb5bf275d440181ae197aeebb069403d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T02:08:49.005 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:48 smithi103 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.0.service: Deactivated successfully. 2024-09-16T02:08:49.005 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:48 smithi103 systemd[1]: Stopped Ceph osd.0 for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:08:49.005 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 16 02:08:48 smithi103 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.0.service: Consumed 13.042s CPU time. 2024-09-16T02:08:49.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:49.015 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T02:08:49.015 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T02:08:49.015 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.1 2024-09-16T02:08:49.343 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:49 smithi136 systemd[1]: Stopping Ceph osd.1 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:49.773 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:49 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1[39801]: 2024-09-16T02:08:49.342+0000 7f78d6215640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:49.773 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:49 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1[39801]: 2024-09-16T02:08:49.342+0000 7f78d6215640 -1 osd.1 84 *** Got signal Terminated *** 2024-09-16T02:08:49.773 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:49 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1[39801]: 2024-09-16T02:08:49.342+0000 7f78d6215640 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:08:55.143 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:54 smithi136 podman[69002]: 2024-09-16 02:08:54.879461217 +0000 UTC m=+5.661427526 container died b89abf5a26da2914cdaee7665c156eeb881210e56a97cec98c102da1e05eadbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T02:08:55.523 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:55 smithi136 podman[69002]: 2024-09-16 02:08:55.216129885 +0000 UTC m=+5.998096187 container cleanup b89abf5a26da2914cdaee7665c156eeb881210e56a97cec98c102da1e05eadbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T02:08:55.523 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:55 smithi136 bash[69002]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1 2024-09-16T02:08:55.523 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:55 smithi136 podman[69018]: 2024-09-16 02:08:55.392404409 +0000 UTC m=+0.507273127 container remove b89abf5a26da2914cdaee7665c156eeb881210e56a97cec98c102da1e05eadbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T02:08:56.113 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:55 smithi136 podman[69110]: 2024-09-16 02:08:55.728555447 +0000 UTC m=+0.016078497 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:08:56.113 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:55 smithi136 podman[69110]: 2024-09-16 02:08:55.868020691 +0000 UTC m=+0.155543734 container create f1ca770199b16fd0ccf4232f43156526e81ddb77360d097a3ad745af02e5fba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-16T02:08:56.523 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:56 smithi136 podman[69110]: 2024-09-16 02:08:56.112022574 +0000 UTC m=+0.399545618 container init f1ca770199b16fd0ccf4232f43156526e81ddb77360d097a3ad745af02e5fba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T02:08:56.523 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:56 smithi136 podman[69110]: 2024-09-16 02:08:56.115699432 +0000 UTC m=+0.403222476 container start f1ca770199b16fd0ccf4232f43156526e81ddb77360d097a3ad745af02e5fba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:08:56.523 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:56 smithi136 podman[69110]: 2024-09-16 02:08:56.195294068 +0000 UTC m=+0.482817124 container attach f1ca770199b16fd0ccf4232f43156526e81ddb77360d097a3ad745af02e5fba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T02:08:56.523 INFO:journalctl@ceph.osd.1.smithi136.stdout:Sep 16 02:08:56 smithi136 podman[69110]: 2024-09-16 02:08:56.303873801 +0000 UTC m=+0.591396846 container died f1ca770199b16fd0ccf4232f43156526e81ddb77360d097a3ad745af02e5fba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-16T02:08:56.903 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T02:08:56.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:08:56.944 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T02:08:56.944 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T02:08:56.944 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.2 2024-09-16T02:08:57.295 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:08:56 smithi172 systemd[1]: Stopping Ceph osd.2 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:08:57.795 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:08:57 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2[40339]: 2024-09-16T02:08:57.334+0000 7f6fd16ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:08:57.795 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:08:57 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2[40339]: 2024-09-16T02:08:57.334+0000 7f6fd16ad640 -1 osd.2 84 *** Got signal Terminated *** 2024-09-16T02:08:57.795 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:08:57 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2[40339]: 2024-09-16T02:08:57.334+0000 7f6fd16ad640 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:09:03.137 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:09:02 smithi172 podman[69207]: 2024-09-16 02:09:02.870024582 +0000 UTC m=+5.725248476 container died 5feed3346b0af785fa1d98fdbf1601c483df05341cd0a9d9d9767b33bb7782d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T02:09:03.520 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:09:03 smithi172 podman[69207]: 2024-09-16 02:09:03.234521053 +0000 UTC m=+6.089744946 container cleanup 5feed3346b0af785fa1d98fdbf1601c483df05341cd0a9d9d9767b33bb7782d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-16T02:09:03.520 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:09:03 smithi172 bash[69207]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2 2024-09-16T02:09:03.795 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:09:03 smithi172 podman[69222]: 2024-09-16 02:09:03.519148485 +0000 UTC m=+0.646057339 container remove 5feed3346b0af785fa1d98fdbf1601c483df05341cd0a9d9d9767b33bb7782d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T02:09:04.082 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:09:03 smithi172 podman[69311]: 2024-09-16 02:09:03.729807963 +0000 UTC m=+0.015447920 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:09:04.082 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:09:03 smithi172 podman[69311]: 2024-09-16 02:09:03.853579739 +0000 UTC m=+0.139219731 container create 420ef6684370c98d19757568e5efef4838565545f5a564717c24a7aa66f9ef2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T02:09:04.082 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:09:04 smithi172 podman[69311]: 2024-09-16 02:09:04.081023504 +0000 UTC m=+0.366663460 container init 420ef6684370c98d19757568e5efef4838565545f5a564717c24a7aa66f9ef2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:09:04.545 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:09:04 smithi172 podman[69311]: 2024-09-16 02:09:04.0847136 +0000 UTC m=+0.370353548 container start 420ef6684370c98d19757568e5efef4838565545f5a564717c24a7aa66f9ef2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-16T02:09:04.545 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:09:04 smithi172 podman[69311]: 2024-09-16 02:09:04.164299902 +0000 UTC m=+0.449939858 container attach 420ef6684370c98d19757568e5efef4838565545f5a564717c24a7aa66f9ef2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T02:09:04.545 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 16 02:09:04 smithi172 podman[69334]: 2024-09-16 02:09:04.303916953 +0000 UTC m=+0.016991618 container died 420ef6684370c98d19757568e5efef4838565545f5a564717c24a7aa66f9ef2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-16T02:09:04.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T02:09:04.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:04.923 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T02:09:04.923 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T02:09:04.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.3 2024-09-16T02:09:05.255 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:05 smithi103 systemd[1]: Stopping Ceph osd.3 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:09:05.755 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:05 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3[51202]: 2024-09-16T02:09:05.373+0000 7fed4bc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:09:05.755 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:05 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3[51202]: 2024-09-16T02:09:05.373+0000 7fed4bc00640 -1 osd.3 84 *** Got signal Terminated *** 2024-09-16T02:09:05.755 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:05 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3[51202]: 2024-09-16T02:09:05.373+0000 7fed4bc00640 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:09:07.505 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3[51202]: 2024-09-16T02:09:07.247+0000 7fed47600640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:45.529203+0000 front 2024-09-16T02:08:45.529284+0000 (oldest deadline 2024-09-16T02:09:07.229230+0000) 2024-09-16T02:09:07.505 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:07 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:07.455+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:07.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:07 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:07.307+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:07.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:07 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:07.392+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:08.490 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:08 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3[51202]: 2024-09-16T02:09:08.199+0000 7fed47600640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:45.529203+0000 front 2024-09-16T02:08:45.529284+0000 (oldest deadline 2024-09-16T02:09:07.229230+0000) 2024-09-16T02:09:08.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:08 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:08.274+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:08.729 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:08 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:08.356+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:08.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:08 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:08.488+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:09.505 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:09 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3[51202]: 2024-09-16T02:09:09.216+0000 7fed47600640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:45.529203+0000 front 2024-09-16T02:08:45.529284+0000 (oldest deadline 2024-09-16T02:09:07.229230+0000) 2024-09-16T02:09:09.542 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:09 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:09.378+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:09.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:09.308+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:09.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:09.152+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:09.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:09.360+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:10.005 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:09 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:09.539+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:10.005 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:09 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:09.653+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:10.507 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:10 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3[51202]: 2024-09-16T02:09:10.254+0000 7fed47600640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:45.529203+0000 front 2024-09-16T02:08:45.529284+0000 (oldest deadline 2024-09-16T02:09:07.229230+0000) 2024-09-16T02:09:10.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:10.189+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:10.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:10.308+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:10.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:10.386+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:10.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:10 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:10.384+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:10.776 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:10 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:10.505+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:10.776 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:10 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:10.628+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:11.104 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:10 smithi103 podman[89001]: 2024-09-16 02:09:10.775171117 +0000 UTC m=+5.558962735 container died 551a78cea4e4356aef9bf2d18245a3632bd0835cf15d11fdd08f9529f5ff99bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T02:09:11.492 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:11 smithi103 podman[89001]: 2024-09-16 02:09:11.288296427 +0000 UTC m=+6.072088047 container cleanup 551a78cea4e4356aef9bf2d18245a3632bd0835cf15d11fdd08f9529f5ff99bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T02:09:11.493 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:11 smithi103 bash[89001]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3 2024-09-16T02:09:11.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:11 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:11.214+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:11.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:11 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:11.308+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:11.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:11 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:11.377+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:11.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:11 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:11.491+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:11.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:11 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:11.678+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:11.755 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:11 smithi103 podman[89018]: 2024-09-16 02:09:11.556844903 +0000 UTC m=+0.767297474 container remove 551a78cea4e4356aef9bf2d18245a3632bd0835cf15d11fdd08f9529f5ff99bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T02:09:11.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:11 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:11.340+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:12.386 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:12 smithi103 podman[89105]: 2024-09-16 02:09:11.998054262 +0000 UTC m=+0.027135586 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:09:12.386 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:12 smithi103 podman[89105]: 2024-09-16 02:09:12.099536384 +0000 UTC m=+0.128617707 container create 6f1b835a26b9872f9047bb44b20c4e79529d4a4712bf94c3b00bf674eb4ca209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T02:09:12.386 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:12 smithi103 podman[89105]: 2024-09-16 02:09:12.3017662 +0000 UTC m=+0.330847524 container init 6f1b835a26b9872f9047bb44b20c4e79529d4a4712bf94c3b00bf674eb4ca209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T02:09:12.386 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:12 smithi103 podman[89105]: 2024-09-16 02:09:12.308389935 +0000 UTC m=+0.337471269 container start 6f1b835a26b9872f9047bb44b20c4e79529d4a4712bf94c3b00bf674eb4ca209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T02:09:12.386 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:12 smithi103 podman[89105]: 2024-09-16 02:09:12.384850991 +0000 UTC m=+0.413932346 container attach 6f1b835a26b9872f9047bb44b20c4e79529d4a4712bf94c3b00bf674eb4ca209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T02:09:12.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:12 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:12.277+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:12.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:12 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:12.207+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:12.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:12 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:12.370+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:12.710 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:12 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:12.497+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:12.710 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:12 smithi103 podman[89105]: 2024-09-16 02:09:12.568881132 +0000 UTC m=+0.597962463 container died 6f1b835a26b9872f9047bb44b20c4e79529d4a4712bf94c3b00bf674eb4ca209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-16T02:09:12.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:12 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:12.324+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:13.004 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:12 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:12.708+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:13.356 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T02:09:13.406 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:13 smithi103 podman[89128]: 2024-09-16 02:09:13.322541842 +0000 UTC m=+0.736195712 container remove 6f1b835a26b9872f9047bb44b20c4e79529d4a4712bf94c3b00bf674eb4ca209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:09:13.406 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:13 smithi103 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.3.service: Deactivated successfully. 2024-09-16T02:09:13.406 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:13 smithi103 systemd[1]: Stopped Ceph osd.3 for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:09:13.406 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 16 02:09:13 smithi103 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.3.service: Consumed 9.421s CPU time. 2024-09-16T02:09:13.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:13.433 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T02:09:13.433 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T02:09:13.433 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.4 2024-09-16T02:09:13.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:13 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:13.271+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:13.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:13 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:13.171+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:13.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:13 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:13.375+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:13.744 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:13 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:13.453+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:13.768 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:13 smithi136 systemd[1]: Stopping Ceph osd.4 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:09:13.768 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:13 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:13.293+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:14.004 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:13 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:13.743+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:14.023 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:13 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4[46574]: 2024-09-16T02:09:13.766+0000 7feaa34e5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:09:14.023 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:13 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4[46574]: 2024-09-16T02:09:13.766+0000 7feaa34e5640 -1 osd.4 84 *** Got signal Terminated *** 2024-09-16T02:09:14.023 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:13 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4[46574]: 2024-09-16T02:09:13.766+0000 7feaa34e5640 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:09:14.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:14 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:14.255+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:14.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:14.221+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:14.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:14.273+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:14.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:14.389+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:14.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:14 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:14.451+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:15.254 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:14 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:14.791+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:15.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:15 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:15.264+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:15.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:15 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:15.184+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:15.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:15 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:15.234+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:15.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:15 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:15.397+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:15.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:15 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:15.477+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:15.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:15 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:15.477+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:16.254 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:15 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:15.795+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:16.523 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:16 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4[46574]: 2024-09-16T02:09:16.175+0000 7fea9faea640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:49.963364+0000 front 2024-09-16T02:08:49.963387+0000 (oldest deadline 2024-09-16T02:09:15.863550+0000) 2024-09-16T02:09:16.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:16 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:16.243+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:16.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:16 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:16.214+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:16.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:16 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:16.191+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:16.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:16 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:16.191+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:16.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:16 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:16.407+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:16.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:16 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:16.446+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:16.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:16 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:16.446+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:17.254 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:16 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:16.805+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:17.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:17 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:17.195+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:17.523 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:17 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4[46574]: 2024-09-16T02:09:17.182+0000 7fea9faea640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:49.963364+0000 front 2024-09-16T02:08:49.963387+0000 (oldest deadline 2024-09-16T02:09:15.863550+0000) 2024-09-16T02:09:17.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:17 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:17.240+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:17.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:17 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:17.149+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:17.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:17 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:17.149+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:17.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:17 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:17.385+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:17.589 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:17 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:17.433+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:17.589 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:17 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:17.433+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:18.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:17 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:17.849+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:18.523 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:18 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4[46574]: 2024-09-16T02:09:18.186+0000 7fea9faea640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:49.963364+0000 front 2024-09-16T02:08:49.963387+0000 (oldest deadline 2024-09-16T02:09:15.863550+0000) 2024-09-16T02:09:18.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:18 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:18.213+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:18.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:18 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:18.336+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:18.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:18 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:18.157+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:18.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:18 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:18.157+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:18.546 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:18 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:18.229+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:18.546 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:18 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:18.229+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:51.959541+0000 front 2024-09-16T02:08:51.959604+0000 (oldest deadline 2024-09-16T02:09:17.859397+0000) 2024-09-16T02:09:18.732 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:18 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:18.415+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:18.732 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:18 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:18.415+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:19.005 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:18 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:18.889+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:19.385 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:19 smithi136 podman[69238]: 2024-09-16 02:09:19.117044924 +0000 UTC m=+5.479804913 container died 6d0510a4df130faf19337f6068d567fd4bab3a62b80b3ce1fc03636268d5330e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-16T02:09:19.385 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:19 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:19.253+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:19.385 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:19 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:19.253+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:19.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:19 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:19.113+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:19.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:19 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:19.113+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:19.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:19 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:19.357+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:19.546 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:19 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:19.234+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:19.546 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:19 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:19.234+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:51.959541+0000 front 2024-09-16T02:08:51.959604+0000 (oldest deadline 2024-09-16T02:09:17.859397+0000) 2024-09-16T02:09:19.659 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:19 smithi136 podman[69238]: 2024-09-16 02:09:19.474273277 +0000 UTC m=+5.837033265 container cleanup 6d0510a4df130faf19337f6068d567fd4bab3a62b80b3ce1fc03636268d5330e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:09:19.659 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:19 smithi136 bash[69238]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4 2024-09-16T02:09:19.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:19 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:19.426+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:19.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:19 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:19.426+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:19.920 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:19 smithi136 podman[69255]: 2024-09-16 02:09:19.658442885 +0000 UTC m=+0.534712427 container remove 6d0510a4df130faf19337f6068d567fd4bab3a62b80b3ce1fc03636268d5330e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T02:09:20.208 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:20 smithi136 podman[69347]: 2024-09-16 02:09:19.98119675 +0000 UTC m=+0.016355161 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:09:20.208 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:20 smithi136 podman[69347]: 2024-09-16 02:09:20.117656484 +0000 UTC m=+0.152814891 container create 1037fa7b666a3743093c3ffdf07b7c824c56bc6443fe7510a509bad8c3518b84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=) 2024-09-16T02:09:20.209 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:19 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:19.918+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:20.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:19 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:19.862+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:20.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:19 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:19.862+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:20.510 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:20 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:20.266+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:20.510 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:20 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:20.266+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:20.511 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:20 smithi136 podman[69347]: 2024-09-16 02:09:20.311687343 +0000 UTC m=+0.346845750 container init 1037fa7b666a3743093c3ffdf07b7c824c56bc6443fe7510a509bad8c3518b84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:09:20.511 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:20 smithi136 podman[69347]: 2024-09-16 02:09:20.315031548 +0000 UTC m=+0.350189954 container start 1037fa7b666a3743093c3ffdf07b7c824c56bc6443fe7510a509bad8c3518b84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True) 2024-09-16T02:09:20.511 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:20 smithi136 podman[69347]: 2024-09-16 02:09:20.386660257 +0000 UTC m=+0.421818673 container attach 1037fa7b666a3743093c3ffdf07b7c824c56bc6443fe7510a509bad8c3518b84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T02:09:20.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:20 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:20.313+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:20.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:20 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:20.195+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:20.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:20 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:20.195+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:51.959541+0000 front 2024-09-16T02:08:51.959604+0000 (oldest deadline 2024-09-16T02:09:17.859397+0000) 2024-09-16T02:09:20.546 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:20 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:20.142+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:20.546 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:20 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:20.142+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:20.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:20 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:20.425+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:20.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:20 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:20.425+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:20.773 INFO:journalctl@ceph.osd.4.smithi136.stdout:Sep 16 02:09:20 smithi136 podman[69347]: 2024-09-16 02:09:20.509743104 +0000 UTC m=+0.544901519 container died 1037fa7b666a3743093c3ffdf07b7c824c56bc6443fe7510a509bad8c3518b84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:09:21.057 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:20 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:20.870+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:21.081 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T02:09:21.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:21.121 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T02:09:21.121 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T02:09:21.121 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.5 2024-09-16T02:09:21.254 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:20 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:20.886+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:21.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:20 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:20.886+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:21.421 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:21 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:21.186+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:21.421 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:21 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:21.186+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:21.421 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:21 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:21.270+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:21.421 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:21 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:21.270+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:21.421 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:21 smithi172 systemd[1]: Stopping Ceph osd.5 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:09:21.422 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:21 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:21.181+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:21.422 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:21 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:21.181+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:51.959541+0000 front 2024-09-16T02:08:51.959604+0000 (oldest deadline 2024-09-16T02:09:17.859397+0000) 2024-09-16T02:09:21.422 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:21 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:21.418+0000 7f9da7e33640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:09:21.422 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:21 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:21.418+0000 7f9da7e33640 -1 osd.5 84 *** Got signal Terminated *** 2024-09-16T02:09:21.422 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:21 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:21.418+0000 7f9da7e33640 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:09:21.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:21 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:21.259+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:21.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:21 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:21.259+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:21.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:21 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:21.378+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:21.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:21 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:21.378+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:22.256 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:21 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:21.882+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:22.256 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:21 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:21.882+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:22.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:21 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:21.873+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:22.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:22 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:22.218+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:22.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:22 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:22.218+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:51.959541+0000 front 2024-09-16T02:08:51.959604+0000 (oldest deadline 2024-09-16T02:09:17.859397+0000) 2024-09-16T02:09:22.546 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:22 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:22.235+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:22.546 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:22 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:22.235+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:22.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:22 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:22.306+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:22.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:22 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:22.306+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:22.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:22 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:22.387+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:22.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:22 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:22.387+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:22.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:22 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:22.300+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:22.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:22 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:22.300+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:23.254 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:22 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:22.898+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:23.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:22 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:22.898+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:23.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:22 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:22.858+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:23.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:23 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:23.280+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:23.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:23 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:23.280+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:23.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:23 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:23.248+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:23.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:23 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:23.248+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:51.959541+0000 front 2024-09-16T02:08:51.959604+0000 (oldest deadline 2024-09-16T02:09:17.859397+0000) 2024-09-16T02:09:23.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:23 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:23.308+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:23.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:23 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:23.308+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:23.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:23 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:23.380+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:23.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:23 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:23.380+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:23.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:23 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:23.282+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:23.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:23 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:23.282+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:24.250 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:23 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:23.812+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:24.251 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:23 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:23.812+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:24.254 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:23 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:23.880+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:24.254 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:23 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:23.880+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:24.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:24 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:24.249+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:24.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:24 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:24.249+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:24.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:24 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:24.307+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:24.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:24 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:24.307+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:24.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:24 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:24.307+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:24.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:24 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:24.269+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:24.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:24 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:24.269+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:51.959541+0000 front 2024-09-16T02:08:51.959604+0000 (oldest deadline 2024-09-16T02:09:17.859397+0000) 2024-09-16T02:09:24.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:24 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:24.353+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:24.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:24 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:24.353+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:24.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:24 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:24.398+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:24.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:24 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:24.398+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:25.023 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:24 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:24.770+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:25.023 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:24 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:24.770+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:25.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:24 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:24.918+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:25.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:24 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:24.918+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:25.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:25.228+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:25.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:25.228+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:25.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:25.228+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:25.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:25.350+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:25.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:25.350+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:25.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:25.350+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:25.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:25.355+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:25.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:25.355+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:25.546 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:25.278+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:25.546 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:25.278+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:51.959541+0000 front 2024-09-16T02:08:51.959604+0000 (oldest deadline 2024-09-16T02:09:17.859397+0000) 2024-09-16T02:09:25.546 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:25 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:25.278+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.761153+0000 front 2024-09-16T02:09:01.761201+0000 (oldest deadline 2024-09-16T02:09:24.661223+0000) 2024-09-16T02:09:25.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:25 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:25.384+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:25.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:25 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:25.384+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:26.023 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:25.759+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:26.023 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:25 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:25.759+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:26.254 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:25 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:25.915+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:26.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:25 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:25.915+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:26.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:25 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:25.915+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:26.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:26 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:26.264+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:26.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:26 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:26.264+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:26.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:26 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:26.264+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:26.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:26 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:26.329+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:26.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:26 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:26.329+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:26.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:26 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:26.329+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:26.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:26 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:26.229+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.358301+0000 front 2024-09-16T02:08:44.358521+0000 (oldest deadline 2024-09-16T02:09:08.458332+0000) 2024-09-16T02:09:26.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:26 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:26.229+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:51.959541+0000 front 2024-09-16T02:08:51.959604+0000 (oldest deadline 2024-09-16T02:09:17.859397+0000) 2024-09-16T02:09:26.545 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:26 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5[47265]: 2024-09-16T02:09:26.229+0000 7f9da4438640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.761153+0000 front 2024-09-16T02:09:01.761201+0000 (oldest deadline 2024-09-16T02:09:24.661223+0000) 2024-09-16T02:09:26.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:26 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:26.354+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:26.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:26 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:26.354+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:26.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:26.430+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:26.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:26.430+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:26.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:26.430+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.099723+0000 front 2024-09-16T02:09:01.099741+0000 (oldest deadline 2024-09-16T02:09:26.399495+0000) 2024-09-16T02:09:27.021 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:26 smithi172 podman[69426]: 2024-09-16 02:09:26.722609552 +0000 UTC m=+5.406536480 container died f8994531df5ea366d92671c0d367e50b1f652bddf224e7d85551299d42922736 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T02:09:27.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:26.942+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:27.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:26.942+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:27.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:26 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:26.942+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:27.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:26 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:26.800+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:27.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:26 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:26.800+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:27.295 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:27 smithi172 podman[69426]: 2024-09-16 02:09:27.119618202 +0000 UTC m=+5.803545123 container cleanup f8994531df5ea366d92671c0d367e50b1f652bddf224e7d85551299d42922736 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:09:27.295 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:27 smithi172 bash[69426]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5 2024-09-16T02:09:27.705 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:27 smithi172 podman[69442]: 2024-09-16 02:09:27.395666307 +0000 UTC m=+0.671655985 container remove f8994531df5ea366d92671c0d367e50b1f652bddf224e7d85551299d42922736 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T02:09:27.706 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:27 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:27.352+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:27.706 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:27 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:27.352+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:27.706 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:27 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:27.352+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:27.706 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:27 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:27.328+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:27.706 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:27 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:27.328+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:27.706 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:27 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:27.328+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:27.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:27.458+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:27.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:27.458+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:27.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:27.458+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.099723+0000 front 2024-09-16T02:09:01.099741+0000 (oldest deadline 2024-09-16T02:09:26.399495+0000) 2024-09-16T02:09:27.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:27 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:27.301+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:27.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:27 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:27.301+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:27.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:27 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:27.301+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:27.999 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:27 smithi172 podman[69533]: 2024-09-16 02:09:27.607123266 +0000 UTC m=+0.015818899 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:09:28.000 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:27 smithi172 podman[69533]: 2024-09-16 02:09:27.704736609 +0000 UTC m=+0.113432281 container create cb0b5c8d074be7f205e89a3b41b5219b9978ddfaa3bc1d9d65e399ac154b2540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T02:09:28.000 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:27 smithi172 podman[69533]: 2024-09-16 02:09:27.915596105 +0000 UTC m=+0.324291729 container init cb0b5c8d074be7f205e89a3b41b5219b9978ddfaa3bc1d9d65e399ac154b2540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T02:09:28.000 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:27 smithi172 podman[69533]: 2024-09-16 02:09:27.919001599 +0000 UTC m=+0.327697225 container start cb0b5c8d074be7f205e89a3b41b5219b9978ddfaa3bc1d9d65e399ac154b2540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:09:28.000 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:27 smithi172 podman[69533]: 2024-09-16 02:09:27.998626309 +0000 UTC m=+0.407321936 container attach cb0b5c8d074be7f205e89a3b41b5219b9978ddfaa3bc1d9d65e399ac154b2540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:09:28.225 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:27.942+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:28.225 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:27.942+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:28.225 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:27 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:27.942+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:28.265 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:27 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:27.797+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:28.266 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:27 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:27.797+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:28.295 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 16 02:09:28 smithi172 podman[69533]: 2024-09-16 02:09:28.112221965 +0000 UTC m=+0.520917597 container died cb0b5c8d074be7f205e89a3b41b5219b9978ddfaa3bc1d9d65e399ac154b2540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-16T02:09:28.504 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:28 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:28.460+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:28.505 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:28 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:28.460+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:28.505 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:28 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:28.460+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.099723+0000 front 2024-09-16T02:09:01.099741+0000 (oldest deadline 2024-09-16T02:09:26.399495+0000) 2024-09-16T02:09:28.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:28 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:28.264+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:28.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:28 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:28.264+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:28.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:28 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:28.264+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:28.695 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:28.372+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:28.695 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:28.372+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:28.695 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:28.372+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:28.695 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:28.300+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:28.695 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:28.300+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:28.695 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:28 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:28.300+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:28.715 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T02:09:28.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:28.753 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T02:09:28.753 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T02:09:28.753 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.6 2024-09-16T02:09:29.023 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:28 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:28.770+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:29.023 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:28 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:28.770+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:29.163 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:28 smithi103 systemd[1]: Stopping Ceph osd.6 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:09:29.163 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:28 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:28.942+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:29.163 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:28 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:28.942+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:29.163 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:28 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:28.942+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:29.462 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:29.161+0000 7f3645000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:09:29.462 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:29.161+0000 7f3645000640 -1 osd.6 84 *** Got signal Terminated *** 2024-09-16T02:09:29.462 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:29.161+0000 7f3645000640 -1 osd.6 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:09:29.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:29 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:29.254+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:29.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:29 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:29.254+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:29.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:29 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:29.254+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:29.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:29.461+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:29.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:29.461+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:29.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:29.461+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.099723+0000 front 2024-09-16T02:09:01.099741+0000 (oldest deadline 2024-09-16T02:09:26.399495+0000) 2024-09-16T02:09:29.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:29 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:29.385+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:29.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:29 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:29.385+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:29.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:29 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:29.385+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:29.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:29 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:29.340+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:29.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:29 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:29.340+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:29.796 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:29 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:29.340+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:30.249 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:29 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:29.775+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:30.249 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:29 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:29.775+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:30.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:29.966+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:30.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:29.966+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:30.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:29 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:29.966+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:30.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:30.248+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:30.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:30.248+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:30.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:30.248+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:30.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:30 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:30.478+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:30.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:30 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:30.478+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:30.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:30 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:30.478+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.099723+0000 front 2024-09-16T02:09:01.099741+0000 (oldest deadline 2024-09-16T02:09:26.399495+0000) 2024-09-16T02:09:30.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:30 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:30.340+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:30.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:30 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:30.340+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:30.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:30 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:30.340+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:30.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:30 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:30.297+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:30.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:30 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:30.297+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:30.796 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:30 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:30.297+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:31.023 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:30.758+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:31.023 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:30 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:30.758+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:31.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:30 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:30.994+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:31.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:30 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:30.994+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:31.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:30 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:30.994+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:31.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:31 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:31.244+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:31.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:31 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:31.244+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:31.524 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:31 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:31.244+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:31.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:31 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:31.437+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:31.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:31 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:31.437+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:31.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:31 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:31.437+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.099723+0000 front 2024-09-16T02:09:01.099741+0000 (oldest deadline 2024-09-16T02:09:26.399495+0000) 2024-09-16T02:09:31.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:31.364+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:31.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:31.364+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:31.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:31.364+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:31.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:31.319+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:31.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:31.319+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:31.796 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:31 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:31.319+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:32.245 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:31 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:31.783+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:32.246 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:31 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:31.783+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:32.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:32.039+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:32.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:32.039+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:32.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:32.039+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:32.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:32.244+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:32.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:32.244+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:32.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:32.244+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:32.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:32.244+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:32.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:32.419+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:32.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:32.420+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:32.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:32.420+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.099723+0000 front 2024-09-16T02:09:01.099741+0000 (oldest deadline 2024-09-16T02:09:26.399495+0000) 2024-09-16T02:09:32.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:32 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:32.420+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.399864+0000 front 2024-09-16T02:09:06.400070+0000 (oldest deadline 2024-09-16T02:09:32.299894+0000) 2024-09-16T02:09:32.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:32 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:32.359+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:32.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:32 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:32.359+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:32.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:32 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:32.359+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:32.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:32 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:32.327+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:32.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:32 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:32.327+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:32.796 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:32 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:32.327+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:33.254 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:32.811+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:33.254 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:32.811+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:33.424 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:33 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:33.084+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:33.424 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:33 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:33.084+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:33.424 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:33 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:33.084+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:33.424 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:33 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:33.084+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:33.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:33.253+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:33.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:33.253+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:33.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:33.253+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:33.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:33.253+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:33.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:33 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:33.421+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.994916+0000 front 2024-09-16T02:08:44.994955+0000 (oldest deadline 2024-09-16T02:09:07.195945+0000) 2024-09-16T02:09:33.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:33 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:33.421+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.598181+0000 front 2024-09-16T02:08:53.598096+0000 (oldest deadline 2024-09-16T02:09:15.297951+0000) 2024-09-16T02:09:33.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:33 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:33.421+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.099723+0000 front 2024-09-16T02:09:01.099741+0000 (oldest deadline 2024-09-16T02:09:26.399495+0000) 2024-09-16T02:09:33.755 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:33 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6[58188]: 2024-09-16T02:09:33.421+0000 7f3640a00640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.399864+0000 front 2024-09-16T02:09:06.400070+0000 (oldest deadline 2024-09-16T02:09:32.299894+0000) 2024-09-16T02:09:33.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:33 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:33.373+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:33.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:33 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:33.373+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:33.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:33 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:33.373+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:33.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:33 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:33.296+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:33.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:33 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:33.296+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:33.796 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:33 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:33.296+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:34.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:33.856+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:34.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:33 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:33.856+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:34.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:34 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:34.110+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:34.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:34 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:34.110+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:34.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:34 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:34.110+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:34.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:34 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:34.110+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:34.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:34 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:34.369+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:34.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:34 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:34.369+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:34.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:34 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:34.369+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:34.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:34 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:34.274+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:34.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:34 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:34.274+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:34.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:34 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:34.274+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:34.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:34 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:34.290+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:34.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:34 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:34.290+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:34.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:34 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:34.290+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:34.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:34 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:34.290+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:35.005 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:34 smithi103 podman[89219]: 2024-09-16 02:09:34.717062047 +0000 UTC m=+5.677443439 container stop bad4a2358801a9aed1b00922bde693e457848ed4aa5b171bad8c647631a89f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T02:09:35.005 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:34 smithi103 podman[89219]: 2024-09-16 02:09:34.743004913 +0000 UTC m=+5.703386305 container died bad4a2358801a9aed1b00922bde693e457848ed4aa5b171bad8c647631a89f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:09:35.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:34 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:34.813+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:35.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:34 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:34.813+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:35.436 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:35 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:35.142+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:35.436 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:35 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:35.142+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:35.437 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:35 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:35.142+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:35.437 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:35 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:35.142+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:35.437 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:35 smithi103 podman[89219]: 2024-09-16 02:09:35.258329309 +0000 UTC m=+6.218710700 container cleanup bad4a2358801a9aed1b00922bde693e457848ed4aa5b171bad8c647631a89f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:09:35.437 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:35 smithi103 bash[89219]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6 2024-09-16T02:09:35.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:35 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:35.394+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:35.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:35 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:35.394+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:35.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:35 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:35.394+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:35.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:35 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:35.228+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:35.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:35 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:35.228+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:35.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:35 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:35.228+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:35.754 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:35 smithi103 podman[89238]: 2024-09-16 02:09:35.434401309 +0000 UTC m=+0.696749741 container remove bad4a2358801a9aed1b00922bde693e457848ed4aa5b171bad8c647631a89f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:09:35.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:35 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:35.307+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:35.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:35 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:35.307+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:35.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:35 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:35.307+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:35.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:35 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:35.307+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:36.254 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:35 smithi103 podman[89324]: 2024-09-16 02:09:35.881264533 +0000 UTC m=+0.027643824 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:09:36.255 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:35 smithi103 podman[89324]: 2024-09-16 02:09:35.993781148 +0000 UTC m=+0.140160428 container create 4c8a62c42ebebdeb103a310befcaad6d660f6706826987f5ef5a5f326263eea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T02:09:36.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:36 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:36.178+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:36.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:36 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:36.178+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:36.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:36 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:36.178+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:36.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:36 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:36.178+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:36.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:35 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:35.835+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:36.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:35 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:35.835+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:36.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:35 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:35.835+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:09.565460+0000 front 2024-09-16T02:09:09.565535+0000 (oldest deadline 2024-09-16T02:09:35.465305+0000) 2024-09-16T02:09:36.542 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:36 smithi103 podman[89324]: 2024-09-16 02:09:36.268417095 +0000 UTC m=+0.414796395 container init 4c8a62c42ebebdeb103a310befcaad6d660f6706826987f5ef5a5f326263eea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T02:09:36.542 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:36 smithi103 podman[89324]: 2024-09-16 02:09:36.275287449 +0000 UTC m=+0.421666730 container start 4c8a62c42ebebdeb103a310befcaad6d660f6706826987f5ef5a5f326263eea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:09:36.542 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:36 smithi103 podman[89324]: 2024-09-16 02:09:36.346350416 +0000 UTC m=+0.492729699 container attach 4c8a62c42ebebdeb103a310befcaad6d660f6706826987f5ef5a5f326263eea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T02:09:36.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:36 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:36.406+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:36.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:36 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:36.406+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:36.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:36 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:36.406+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:36.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:36 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:36.277+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:36.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:36 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:36.277+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:36.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:36 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:36.277+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:36.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:36 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:36.302+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:36.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:36 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:36.302+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:36.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:36 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:36.302+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:36.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:36 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:36.302+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:36.883 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 16 02:09:36 smithi103 podman[89324]: 2024-09-16 02:09:36.541667166 +0000 UTC m=+0.688046461 container died 4c8a62c42ebebdeb103a310befcaad6d660f6706826987f5ef5a5f326263eea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-16T02:09:37.192 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T02:09:37.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:37.268 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T02:09:37.269 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T02:09:37.269 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.7 2024-09-16T02:09:37.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:36 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:36.841+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:37.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:36 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:36.841+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:37.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:36 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:36.841+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:09.565460+0000 front 2024-09-16T02:09:09.565535+0000 (oldest deadline 2024-09-16T02:09:35.465305+0000) 2024-09-16T02:09:37.504 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:37 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:37.174+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:37.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:37 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:37.174+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:37.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:37 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:37.174+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:37.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:37 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:37.174+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:37.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:37 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:37.403+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:37.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:37 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:37.403+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:37.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:37 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:37.403+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:37.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:37 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:37.265+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:37.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:37 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:37.265+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:37.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:37 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:37.265+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:37.566 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:37 smithi136 systemd[1]: Stopping Ceph osd.7 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:09:37.567 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:37 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:37.304+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:37.567 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:37 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:37.304+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:37.567 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:37 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:37.304+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:37.567 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:37 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:37.304+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:37.867 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:37 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:37.565+0000 7f7b4294e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:09:37.868 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:37 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:37.565+0000 7f7b4294e640 -1 osd.7 84 *** Got signal Terminated *** 2024-09-16T02:09:37.868 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:37 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:37.565+0000 7f7b4294e640 -1 osd.7 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:09:38.264 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:37 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:37.866+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:38.265 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:37 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:37.866+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:38.265 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:37 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:37.866+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:09.565460+0000 front 2024-09-16T02:09:09.565535+0000 (oldest deadline 2024-09-16T02:09:35.465305+0000) 2024-09-16T02:09:38.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:38 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:38.191+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:38.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:38 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:38.191+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:38.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:38 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:38.191+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:38.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:38 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:38.191+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:38.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:38 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:38.262+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:38.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:38 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:38.262+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:38.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:38 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:38.262+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:38.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:38 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:38.262+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:38.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:38 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:38.358+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:38.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:38 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:38.358+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:38.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:38 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:38.358+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:38.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:38 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:38.219+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:38.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:38 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:38.219+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:38.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:38 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:38.219+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:39.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:38 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:38.844+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:39.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:38 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:38.844+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:39.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:38 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:38.844+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:09.565460+0000 front 2024-09-16T02:09:09.565535+0000 (oldest deadline 2024-09-16T02:09:35.465305+0000) 2024-09-16T02:09:39.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:39 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:39.223+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:39.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:39 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:39.223+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:39.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:39 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:39.223+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:39.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:39 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:39.223+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:39.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:39 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:39.332+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:39.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:39 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:39.332+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:39.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:39 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:39.332+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:39.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:39 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:39.225+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:39.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:39 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:39.225+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:39.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:39 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:39.225+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:39.556 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:39 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:39.292+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:39.556 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:39 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:39.292+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:39.556 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:39 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:39.292+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:39.556 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:39 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:39.292+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:40.270 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:39 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:39.851+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:40.270 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:39 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:39.851+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:40.270 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:39 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:39.851+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:09.565460+0000 front 2024-09-16T02:09:09.565535+0000 (oldest deadline 2024-09-16T02:09:35.465305+0000) 2024-09-16T02:09:40.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:40 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:40.174+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:40.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:40 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:40.174+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:40.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:40 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:40.174+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:40.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:40 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:40.174+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:40.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:40 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:40.174+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:40.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:40 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:40.267+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:40.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:40 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:40.267+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:40.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:40 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:40.267+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:40.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:40 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:40.267+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:40.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:40 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:40.365+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:40.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:40 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:40.365+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:40.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:40 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:40.365+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:40.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:40 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:40.365+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:40.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:40 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:40.204+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:40.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:40 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:40.204+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:40.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:40 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:40.204+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:40.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:40 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:40.204+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:41.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:40 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:40.807+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:41.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:40 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:40.807+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:41.273 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:40 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:40.807+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:09.565460+0000 front 2024-09-16T02:09:09.565535+0000 (oldest deadline 2024-09-16T02:09:35.465305+0000) 2024-09-16T02:09:41.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:41 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:41.169+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:41.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:41 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:41.169+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:41.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:41 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:41.169+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:41.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:41 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:41.169+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:41.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:41 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:41.169+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:41.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:41 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:41.379+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:41.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:41 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:41.379+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:41.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:41 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:41.379+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:41.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:41 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:41.379+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:41.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:41 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:41.200+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:41.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:41 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:41.200+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:41.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:41 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:41.200+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:41.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:41 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:41.200+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:41.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:41 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:41.297+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:41.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:41 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:41.297+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:41.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:41 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:41.297+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:41.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:41 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:41.297+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:42.258 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:41 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:41.780+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.962992+0000 front 2024-09-16T02:08:48.962973+0000 (oldest deadline 2024-09-16T02:09:19.563923+0000) 2024-09-16T02:09:42.264 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:41 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:41.780+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.564314+0000 front 2024-09-16T02:08:59.564413+0000 (oldest deadline 2024-09-16T02:09:23.664445+0000) 2024-09-16T02:09:42.265 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:41 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7[53521]: 2024-09-16T02:09:41.780+0000 7f7b3af4b640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:09.565460+0000 front 2024-09-16T02:09:09.565535+0000 (oldest deadline 2024-09-16T02:09:35.465305+0000) 2024-09-16T02:09:42.377 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:42 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:42.374+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:42.377 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:42 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:42.374+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:42.377 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:42 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:42.226+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:42.377 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:42 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:42.226+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:42.377 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:42 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:42.226+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:42.377 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:42 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:42.226+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:42.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:42 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:42.127+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:42.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:42 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:42.127+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:42.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:42 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:42.127+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:42.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:42 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:42.127+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:42.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:42 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:42.127+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:42.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:42 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:42.256+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:42.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:42 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:42.256+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:42.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:42 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:42.256+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:42.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:42 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:42.256+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:42.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:42 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:42.256+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:42.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:42 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:42.374+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:42.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:42 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:42.374+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:43.317 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:43 smithi136 podman[69450]: 2024-09-16 02:09:43.062061038 +0000 UTC m=+5.600535973 container stop 0871134013f0411c14769a4afa615fdc0ad96457d517ba4d7f681d50732b72a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T02:09:43.318 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:43 smithi136 podman[69450]: 2024-09-16 02:09:43.067399103 +0000 UTC m=+5.605874056 container died 0871134013f0411c14769a4afa615fdc0ad96457d517ba4d7f681d50732b72a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T02:09:43.318 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:43 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:43.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:43.318 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:43 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:43.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:43.318 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:43 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:43.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:43.318 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:43 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:43.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:43.318 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:43 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:43.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:43.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:43 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:43.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:43.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:43 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:43.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:43.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:43 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:43.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:43.506 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:43 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:43.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:43.506 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:43 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:43.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:43.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:43 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:43.347+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:43.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:43 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:43.347+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:43.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:43 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:43.347+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:43.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:43 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:43.347+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:43.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:43 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:43.180+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:43.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:43 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:43.180+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:43.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:43 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:43.180+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:43.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:43 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:43.180+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:43.600 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:43 smithi136 podman[69450]: 2024-09-16 02:09:43.406579924 +0000 UTC m=+5.945054857 container cleanup 0871134013f0411c14769a4afa615fdc0ad96457d517ba4d7f681d50732b72a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T02:09:43.600 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:43 smithi136 bash[69450]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7 2024-09-16T02:09:44.021 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:43 smithi136 podman[69479]: 2024-09-16 02:09:43.599294222 +0000 UTC m=+0.524598163 container remove 0871134013f0411c14769a4afa615fdc0ad96457d517ba4d7f681d50732b72a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T02:09:44.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:44 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:44.260+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:44.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:44 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:44.260+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:44.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:44 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:44.260+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:44.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:44 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:44.260+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:44.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:44 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:44.260+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:44.274 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:44 smithi136 podman[69570]: 2024-09-16 02:09:43.923315187 +0000 UTC m=+0.016124404 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:09:44.274 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:44 smithi136 podman[69570]: 2024-09-16 02:09:44.033423981 +0000 UTC m=+0.126233220 container create 271be4208b8572ae64155b4ba18b58219659c86a6240d1d8d3176baf42a9ed5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T02:09:44.274 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:44 smithi136 podman[69570]: 2024-09-16 02:09:44.260607742 +0000 UTC m=+0.353416938 container init 271be4208b8572ae64155b4ba18b58219659c86a6240d1d8d3176baf42a9ed5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T02:09:44.274 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:44 smithi136 podman[69570]: 2024-09-16 02:09:44.26396615 +0000 UTC m=+0.356775346 container start 271be4208b8572ae64155b4ba18b58219659c86a6240d1d8d3176baf42a9ed5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T02:09:44.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:44 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:44.175+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:44.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:44 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:44.175+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:44.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:44 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:44.175+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:44.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:44 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:44.175+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:44.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:44 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:44.175+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:44.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:44 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:44.389+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:44.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:44 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:44.389+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:44.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:44 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:44.389+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:44.546 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:44 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:44.389+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:44.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:44 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:44.211+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:44.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:44 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:44.211+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:44.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:44 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:44.211+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:44.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:44 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:44.211+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:44.747 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:44 smithi136 podman[69570]: 2024-09-16 02:09:44.360778417 +0000 UTC m=+0.453587614 container attach 271be4208b8572ae64155b4ba18b58219659c86a6240d1d8d3176baf42a9ed5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T02:09:44.747 INFO:journalctl@ceph.osd.7.smithi136.stdout:Sep 16 02:09:44 smithi136 podman[69570]: 2024-09-16 02:09:44.459922123 +0000 UTC m=+0.552731326 container died 271be4208b8572ae64155b4ba18b58219659c86a6240d1d8d3176baf42a9ed5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T02:09:45.030 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T02:09:45.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:45.112 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T02:09:45.112 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-16T02:09:45.112 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.8 2024-09-16T02:09:45.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:45 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:45.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:45.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:45 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:45.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:45.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:45 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:45.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:45.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:45 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:45.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:45.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:45 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:45.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:45.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:45 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:45.203+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:45.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:45 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:45.203+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:45.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:45 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:45.203+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:45.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:45 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:45.203+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:45.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:45 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:45.203+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:45.513 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:45.208+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:45.513 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:45.208+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:45.513 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:45.208+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:45.513 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:45.208+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:45.514 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:45 smithi172 systemd[1]: Stopping Ceph osd.8 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:09:45.514 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:45.374+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:45.514 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:45.374+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:45.514 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:45.374+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:45.514 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:45.374+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:45.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:45.512+0000 7f73b4a4a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:09:45.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:45.512+0000 7f73b4a4a640 -1 osd.8 84 *** Got signal Terminated *** 2024-09-16T02:09:45.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:45 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:45.512+0000 7f73b4a4a640 -1 osd.8 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:09:46.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:46 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:46.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:46.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:46 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:46.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:46.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:46 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:46.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:46.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:46 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:46.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:46.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:46 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:46.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:46.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:46 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:46.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:46.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:46 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:46.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:46.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:46 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:46.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:46.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:46 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:46.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:46.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:46 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:46.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:46.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:46 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:46.380+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:46.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:46 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:46.380+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:46.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:46 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:46.380+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:46.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:46 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:46.380+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:46.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:46 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:46.199+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:46.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:46 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:46.199+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:46.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:46 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:46.199+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:46.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:46 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:46.199+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:47.406 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:47 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:47.178+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:47.406 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:47 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:47.178+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:47.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:47 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:47.178+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:47.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:47 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:47.178+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:47.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:47 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:47.178+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:47.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:47 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:47.183+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:47.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:47 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:47.183+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:47.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:47 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:47.183+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:47.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:47 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:47.183+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:47.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:47 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:47.183+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:47.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:47 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:47.391+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:47.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:47 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:47.391+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:47.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:47 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:47.391+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:47.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:47 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:47.391+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:47.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:47 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:47.216+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:47.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:47 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:47.216+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:47.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:47 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:47.216+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:47.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:47 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:47.216+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:47.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:47 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:47.216+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:48.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:48 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:48.199+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:48.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:48 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:48.199+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:48.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:48 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:48.199+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:48.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:48 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:48.199+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:48.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:48 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:48.199+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:48.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:48 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:48.199+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:48.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:48 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:48.161+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:48.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:48 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:48.161+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:48.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:48 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:48.161+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:48.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:48 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:48.161+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:48.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:48 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:48.161+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:48.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:48.376+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:48.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:48.376+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:48.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:48.376+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:48.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:48.376+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:48.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:48.180+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:48.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:48.180+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:48.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:48.180+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:48.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:48.180+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:48.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:48 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:48.180+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:49.424 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:49.146+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:49.425 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:49.146+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:49.425 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:49.146+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:49.425 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:49.146+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:49.425 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:49.146+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:49.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:49 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:49.176+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:49.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:49 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:49.176+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:49.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:49 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:49.176+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:49.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:49 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:49.176+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:49.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:49 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:49.176+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:49.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:49 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:49.176+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:49.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:49 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:49.209+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:49.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:49 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:49.209+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:49.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:49 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:49.209+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:49.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:49 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:49.209+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:49.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:49 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:49.209+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:49.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:49 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:49.209+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:49.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:49.423+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:49.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:49.423+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:49.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:49.423+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:49.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:49.423+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:50.409 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:50 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:50.150+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:50.410 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:50 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:50.150+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:50.410 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:50 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:50.150+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:50.410 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:50 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:50.150+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:50.410 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:50 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:50.150+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:50.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:50.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:50.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:50.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:50.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:50.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:50.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:50.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:50.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:50.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:50.506 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:50 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:50.196+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:50.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:50 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:50.248+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:50.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:50 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:50.248+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:50.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:50 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:50.248+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:50.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:50 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:50.248+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:50.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:50 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:50.248+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:50.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:50 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:50.248+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:50.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:50 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:50.408+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:42.881707+0000 front 2024-09-16T02:08:42.881787+0000 (oldest deadline 2024-09-16T02:09:06.981678+0000) 2024-09-16T02:09:50.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:50 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:50.408+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:53.383467+0000 front 2024-09-16T02:08:53.383447+0000 (oldest deadline 2024-09-16T02:09:15.683451+0000) 2024-09-16T02:09:50.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:50 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:50.408+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.584529+0000 front 2024-09-16T02:08:58.584620+0000 (oldest deadline 2024-09-16T02:09:23.284668+0000) 2024-09-16T02:09:50.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:50 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:50.408+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:17.287226+0000 front 2024-09-16T02:09:17.287318+0000 (oldest deadline 2024-09-16T02:09:40.187176+0000) 2024-09-16T02:09:50.795 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:50 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8[54244]: 2024-09-16T02:09:50.408+0000 7f73ad047640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.688690+0000 front 2024-09-16T02:09:24.688765+0000 (oldest deadline 2024-09-16T02:09:49.988886+0000) 2024-09-16T02:09:51.238 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:50 smithi172 podman[69635]: 2024-09-16 02:09:50.977265938 +0000 UTC m=+5.669214566 container died f315adbd1f6cb6578975834e1dd6bfdc8f38618ab11a3daa028ce63dcf2ce6b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T02:09:51.239 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:51 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:51.173+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:51.239 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:51 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:51.173+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:51.239 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:51 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:51.173+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:51.239 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:51 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:51.173+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:51.239 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:51 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:51.173+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:51.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:51 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:51.242+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:51.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:51 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:51.242+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:51.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:51 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:51.242+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:51.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:51 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:51.242+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:51.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:51 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:51.242+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:51.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:51 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:51.242+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:51.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:51 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:51.233+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:51.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:51 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:51.233+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:51.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:51 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:51.233+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:51.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:51 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:51.233+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:51.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:51 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:51.233+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:51.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:51 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:51.233+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:51.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:51 smithi172 podman[69635]: 2024-09-16 02:09:51.353459688 +0000 UTC m=+6.045408312 container cleanup f315adbd1f6cb6578975834e1dd6bfdc8f38618ab11a3daa028ce63dcf2ce6b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:09:51.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:51 smithi172 bash[69635]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8 2024-09-16T02:09:51.917 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:51 smithi172 podman[69652]: 2024-09-16 02:09:51.60463609 +0000 UTC m=+0.629563585 container remove f315adbd1f6cb6578975834e1dd6bfdc8f38618ab11a3daa028ce63dcf2ce6b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T02:09:52.170 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:51 smithi172 podman[69742]: 2024-09-16 02:09:51.819326693 +0000 UTC m=+0.015727677 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:09:52.170 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:51 smithi172 podman[69742]: 2024-09-16 02:09:51.938603196 +0000 UTC m=+0.135004223 container create 1e0e0c23bff42bd156f397e107f63b33f67b337b436cc4f60279f0c621ccbf2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T02:09:52.170 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:52 smithi172 podman[69742]: 2024-09-16 02:09:52.166223108 +0000 UTC m=+0.362624092 container init 1e0e0c23bff42bd156f397e107f63b33f67b337b436cc4f60279f0c621ccbf2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T02:09:52.171 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:52 smithi172 podman[69742]: 2024-09-16 02:09:52.169546124 +0000 UTC m=+0.365947120 container start 1e0e0c23bff42bd156f397e107f63b33f67b337b436cc4f60279f0c621ccbf2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T02:09:52.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:52 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:52.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:52.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:52 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:52.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:52.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:52 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:52.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:52.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:52 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:52.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:52.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:52 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:52.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:52.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:52 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:52.188+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:52.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:52 smithi172 podman[69742]: 2024-09-16 02:09:52.274477224 +0000 UTC m=+0.470878217 container attach 1e0e0c23bff42bd156f397e107f63b33f67b337b436cc4f60279f0c621ccbf2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T02:09:52.545 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 16 02:09:52 smithi172 podman[69742]: 2024-09-16 02:09:52.363495061 +0000 UTC m=+0.559896046 container died 1e0e0c23bff42bd156f397e107f63b33f67b337b436cc4f60279f0c621ccbf2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-8921c8e) 2024-09-16T02:09:52.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:52 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:52.172+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:52.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:52 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:52.172+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:52.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:52 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:52.172+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:52.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:52 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:52.172+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:52.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:52 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:52.172+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:52.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:52.279+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:52.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:52.279+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:52.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:52.279+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:52.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:52.279+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:52.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:52.279+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:52.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:52 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:52.279+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:52.947 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.8.service' 2024-09-16T02:09:52.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:09:52.988 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-16T02:09:52.988 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-16T02:09:52.988 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.9 2024-09-16T02:09:53.290 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:53 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:53.159+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:53.290 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:53 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:53.159+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:53.290 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:53 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:53.159+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:53.290 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:53 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:53.159+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:53.290 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:53 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:53.159+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:53.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:53 smithi103 systemd[1]: Stopping Ceph osd.9 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:09:53.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:53 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:53.242+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:53.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:53 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:53.242+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:53.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:53 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:53.242+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:53.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:53 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:53.242+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:53.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:53 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:53.243+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:53.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:53 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:53.243+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:53.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:53 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:53.200+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:53.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:53 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:53.200+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:53.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:53 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:53.200+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:53.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:53 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:53.200+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:53.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:53 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:53.200+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:53.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:53 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:53.200+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:53.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:53 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:53.404+0000 7f5448200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:09:53.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:53 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:53.404+0000 7f5448200640 -1 osd.9 84 *** Got signal Terminated *** 2024-09-16T02:09:53.755 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:53 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:53.404+0000 7f5448200640 -1 osd.9 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:09:54.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:54 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:54.200+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:54.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:54 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:54.200+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:54.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:54 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:54.200+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:54.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:54 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:54.200+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:54.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:54 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:54.200+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:54.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:54 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:54.200+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:54.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:54 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:54.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:54.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:54 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:54.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:54.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:54 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:54.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:54.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:54 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:54.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:54.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:54 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:54.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:54.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:54 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:54.236+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:54.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:54 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:54.115+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:54.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:54 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:54.115+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:54.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:54 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:54.115+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:54.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:54 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:54.115+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:54.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:54 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:54.115+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:55.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:55 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:55.197+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:55.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:55 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:55.197+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:55.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:55 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:55.197+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:55.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:55 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:55.197+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:55.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:55 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:55.197+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:55.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:55 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:55.197+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:55.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:55 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:55.238+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:55.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:55 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:55.238+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:55.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:55 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:55.238+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:55.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:55 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:55.238+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:55.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:55 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:55.238+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:55.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:55 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:55.238+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:55.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:55 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:55.124+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:55.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:55 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:55.124+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:55.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:55 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:55.124+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:55.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:55 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:55.124+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:55.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:55 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:55.124+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:56.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:56.187+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:56.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:56.187+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:56.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:56.187+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:56.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:56.187+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:56.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:56.187+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:56.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:56 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:56.187+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:56.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:56 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:56.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:56.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:56 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:56.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:56.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:56 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:56.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:56.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:56 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:56.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:56.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:56 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:56.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:56.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:56 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:56.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:56.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:56 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:56.082+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:56.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:56 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:56.082+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:56.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:56 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:56.082+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:56.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:56 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:56.082+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:56.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:56 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:56.082+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:57.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:57.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:57.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:57.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:57.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:57.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:57.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:57.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:57.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:57.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:57.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:57 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:57.146+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:57.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:57 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:57.216+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:57.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:57 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:57.216+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:57.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:57 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:57.216+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:57.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:57 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:57.216+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:57.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:57 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:57.216+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:57.524 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:57 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:57.216+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:57.524 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:57 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:57.216+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:32.520559+0000 front 2024-09-16T02:09:32.520488+0000 (oldest deadline 2024-09-16T02:09:56.620394+0000) 2024-09-16T02:09:57.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:57 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:57.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:57.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:57 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:57.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:57.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:57 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:57.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:57.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:57 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:57.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:57.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:57 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:57.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:58.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:58 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:58.043+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:58.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:58 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:58.043+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:58.296 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:58 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:58.043+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:58.296 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:58 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:58.043+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:58.296 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:58 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:58.043+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:58.296 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:58 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:58.043+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:09:58.505 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:58 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:58.099+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.664083+0000 front 2024-09-16T02:08:44.664180+0000 (oldest deadline 2024-09-16T02:09:09.364196+0000) 2024-09-16T02:09:58.509 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:58 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:58.099+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:54.065674+0000 front 2024-09-16T02:08:54.065793+0000 (oldest deadline 2024-09-16T02:09:19.365791+0000) 2024-09-16T02:09:58.509 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:58 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:58.099+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:59.867075+0000 front 2024-09-16T02:08:59.867043+0000 (oldest deadline 2024-09-16T02:09:25.166788+0000) 2024-09-16T02:09:58.509 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:58 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:58.099+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:06.867837+0000 front 2024-09-16T02:09:06.867751+0000 (oldest deadline 2024-09-16T02:09:32.767609+0000) 2024-09-16T02:09:58.509 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:58 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:58.099+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.268692+0000 front 2024-09-16T02:09:16.268733+0000 (oldest deadline 2024-09-16T02:09:39.768514+0000) 2024-09-16T02:09:58.509 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:58 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:58.099+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:25.069743+0000 front 2024-09-16T02:09:25.069811+0000 (oldest deadline 2024-09-16T02:09:47.969517+0000) 2024-09-16T02:09:58.509 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:58 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9[64782]: 2024-09-16T02:09:58.099+0000 7f543ec00640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:33.671325+0000 front 2024-09-16T02:09:33.671377+0000 (oldest deadline 2024-09-16T02:09:57.771437+0000) 2024-09-16T02:09:58.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:58 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:58.225+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:58.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:58 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:58.225+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:58.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:58 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:58.225+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:58.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:58 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:58.225+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:58.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:58 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:58.225+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:58.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:58 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:58.225+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:58.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:58 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:58.225+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:32.520559+0000 front 2024-09-16T02:09:32.520488+0000 (oldest deadline 2024-09-16T02:09:56.620394+0000) 2024-09-16T02:09:59.255 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:58 smithi103 podman[89441]: 2024-09-16 02:09:58.946656551 +0000 UTC m=+5.668630139 container died db171ea63b3919de59ee95e712c56981c83cd0e8aa93b34aa9d0033d9bd33f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T02:09:59.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:59 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:59.239+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:09:59.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:59 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:59.239+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:09:59.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:59 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:59.239+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:09:59.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:59 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:59.239+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:09:59.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:59 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:59.239+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:09:59.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:59 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:59.239+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:09:59.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:09:59 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:09:59.239+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:32.520559+0000 front 2024-09-16T02:09:32.520488+0000 (oldest deadline 2024-09-16T02:09:56.620394+0000) 2024-09-16T02:09:59.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:59 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:59.045+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:09:59.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:59 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:59.045+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:09:59.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:59 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:59.045+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:09:59.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:59 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:59.045+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:09:59.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:59 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:59.045+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:09:59.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:09:59 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:09:59.045+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:09:59.713 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:59 smithi103 podman[89441]: 2024-09-16 02:09:59.459939463 +0000 UTC m=+6.181913050 container cleanup db171ea63b3919de59ee95e712c56981c83cd0e8aa93b34aa9d0033d9bd33f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T02:09:59.714 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:59 smithi103 bash[89441]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9 2024-09-16T02:10:00.004 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:09:59 smithi103 podman[89457]: 2024-09-16 02:09:59.711555447 +0000 UTC m=+0.756577656 container remove db171ea63b3919de59ee95e712c56981c83cd0e8aa93b34aa9d0033d9bd33f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T02:10:00.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:00 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:00.234+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:10:00.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:00 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:00.234+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:10:00.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:00 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:00.234+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:10:00.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:00 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:00.234+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:10:00.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:00 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:00.234+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:10:00.524 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:00 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:00.234+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:10:00.524 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:00 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:00.234+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:32.520559+0000 front 2024-09-16T02:09:32.520488+0000 (oldest deadline 2024-09-16T02:09:56.620394+0000) 2024-09-16T02:10:00.541 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:10:00 smithi103 podman[89544]: 2024-09-16 02:10:00.187725246 +0000 UTC m=+0.027179971 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:10:00.541 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:10:00 smithi103 podman[89544]: 2024-09-16 02:10:00.312119889 +0000 UTC m=+0.151574613 container create 4dfc5a218f80da561736d8b2270f784e38c5cf960a83ff93a131762674e3a144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T02:10:00.541 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:10:00 smithi103 podman[89544]: 2024-09-16 02:10:00.539505117 +0000 UTC m=+0.378959842 container init 4dfc5a218f80da561736d8b2270f784e38c5cf960a83ff93a131762674e3a144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T02:10:00.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:00 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:00.079+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:00.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:00 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:00.079+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:00.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:00 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:00.079+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:00.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:00 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:00.079+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:00.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:00 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:00.079+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:00.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:00 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:00.079+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:00.808 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:10:00 smithi103 podman[89544]: 2024-09-16 02:10:00.546677539 +0000 UTC m=+0.386132271 container start 4dfc5a218f80da561736d8b2270f784e38c5cf960a83ff93a131762674e3a144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_CLEAN=True) 2024-09-16T02:10:00.808 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:10:00 smithi103 podman[89544]: 2024-09-16 02:10:00.622646019 +0000 UTC m=+0.462100744 container attach 4dfc5a218f80da561736d8b2270f784e38c5cf960a83ff93a131762674e3a144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T02:10:01.125 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 16 02:10:00 smithi103 podman[89544]: 2024-09-16 02:10:00.807945945 +0000 UTC m=+0.647400673 container died 4dfc5a218f80da561736d8b2270f784e38c5cf960a83ff93a131762674e3a144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-9-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-16T02:10:01.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:01.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:10:01.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:01.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:10:01.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:01.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:10:01.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:01.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:10:01.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:01.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:10:01.523 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:01.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:10:01.524 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:01.241+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:32.520559+0000 front 2024-09-16T02:09:32.520488+0000 (oldest deadline 2024-09-16T02:09:56.620394+0000) 2024-09-16T02:10:01.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:01 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:01.078+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:01.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:01 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:01.078+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:01.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:01 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:01.078+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:01.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:01 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:01.078+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:01.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:01 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:01.078+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:01.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:01 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:01.078+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:01.593 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.9.service' 2024-09-16T02:10:01.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:01.668 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-16T02:10:01.668 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-16T02:10:01.669 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.10 2024-09-16T02:10:01.974 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 systemd[1]: Stopping Ceph osd.10 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:02.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:01.972+0000 7f6a5307a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:02.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:01.972+0000 7f6a5307a640 -1 osd.10 84 *** Got signal Terminated *** 2024-09-16T02:10:02.273 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:01 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:01.972+0000 7f6a5307a640 -1 osd.10 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:10:02.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:02 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:02.037+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:02.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:02 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:02.037+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:02.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:02 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:02.037+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:02.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:02 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:02.037+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:02.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:02 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:02.037+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:02.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:02 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:02.037+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:02.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:02 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:02.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:10:02.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:02 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:02.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:10:02.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:02 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:02.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:10:02.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:02 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:02.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:10:02.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:02 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:02.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:10:02.774 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:02 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:02.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:10:02.774 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:02 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:02.287+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:32.520559+0000 front 2024-09-16T02:09:32.520488+0000 (oldest deadline 2024-09-16T02:09:56.620394+0000) 2024-09-16T02:10:03.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:03 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:03.061+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:03.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:03 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:03.061+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:03.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:03 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:03.061+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:03.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:03 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:03.061+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:03.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:03 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:03.061+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:03.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:03 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:03.061+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:03.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:03 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:03.322+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:10:03.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:03 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:03.322+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:10:03.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:03 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:03.322+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:10:03.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:03 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:03.322+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:10:03.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:03 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:03.322+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:10:03.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:03 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:03.322+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:10:03.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:03 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:03.322+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:32.520559+0000 front 2024-09-16T02:09:32.520488+0000 (oldest deadline 2024-09-16T02:09:56.620394+0000) 2024-09-16T02:10:04.310 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:04 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:04.044+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:04.310 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:04 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:04.044+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:04.310 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:04 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:04.044+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:04.310 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:04 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:04.044+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:04.310 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:04 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:04.044+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:04.310 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:04 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:04.044+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:04.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:04 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:04.324+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:10:04.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:04 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:04.324+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:10:04.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:04 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:04.324+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:10:04.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:04 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:04.324+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:10:04.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:04 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:04.324+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:10:04.774 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:04 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:04.324+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:10:04.774 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:04 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:04.324+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:32.520559+0000 front 2024-09-16T02:09:32.520488+0000 (oldest deadline 2024-09-16T02:09:56.620394+0000) 2024-09-16T02:10:05.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:05 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:05.022+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:05.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:05 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:05.022+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:05.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:05 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:05.022+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:05.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:05 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:05.022+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:05.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:05 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:05.022+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:05.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:05 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:05.022+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:05.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:05 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:05.313+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:10:05.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:05 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:05.313+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:10:05.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:05 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:05.313+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:10:05.774 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:05 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:05.313+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:10:05.774 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:05 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:05.313+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:10:05.774 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:05 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:05.313+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:10:05.774 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:05 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:05.313+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:32.520559+0000 front 2024-09-16T02:09:32.520488+0000 (oldest deadline 2024-09-16T02:09:56.620394+0000) 2024-09-16T02:10:06.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:06 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:06.054+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:06.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:06 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:06.054+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:06.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:06 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:06.054+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:06.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:06 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:06.054+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:06.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:06 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:06.054+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:06.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:06 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:06.054+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:06.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:06 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:06.327+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:40.612798+0000 front 2024-09-16T02:08:40.612756+0000 (oldest deadline 2024-09-16T02:09:06.512709+0000) 2024-09-16T02:10:06.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:06 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:06.327+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:52.715392+0000 front 2024-09-16T02:08:52.715368+0000 (oldest deadline 2024-09-16T02:09:18.615502+0000) 2024-09-16T02:10:06.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:06 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:06.327+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:08:58.615992+0000 front 2024-09-16T02:08:58.616208+0000 (oldest deadline 2024-09-16T02:09:24.515863+0000) 2024-09-16T02:10:06.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:06 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:06.327+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-16T02:09:07.416995+0000 front 2024-09-16T02:09:07.417140+0000 (oldest deadline 2024-09-16T02:09:31.516991+0000) 2024-09-16T02:10:06.773 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:06 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:06.327+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:15.518399+0000 front 2024-09-16T02:09:15.518446+0000 (oldest deadline 2024-09-16T02:09:41.418492+0000) 2024-09-16T02:10:06.774 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:06 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:06.327+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:26.119375+0000 front 2024-09-16T02:09:26.119409+0000 (oldest deadline 2024-09-16T02:09:48.419333+0000) 2024-09-16T02:10:06.774 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:06 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10[60419]: 2024-09-16T02:10:06.327+0000 7f6a4f67f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:32.520559+0000 front 2024-09-16T02:09:32.520488+0000 (oldest deadline 2024-09-16T02:09:56.620394+0000) 2024-09-16T02:10:07.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:07 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:07.008+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:07.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:07 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:07.008+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:07.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:07 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:07.008+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:07.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:07 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:07.008+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:07.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:07 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:07.008+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:07.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:07 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:07.008+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:07.295 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:07 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:07.008+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-09-16T02:09:41.081448+0000 front 2024-09-16T02:09:41.081511+0000 (oldest deadline 2024-09-16T02:10:06.381391+0000) 2024-09-16T02:10:07.725 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:07 smithi136 podman[69672]: 2024-09-16 02:10:07.427964673 +0000 UTC m=+5.566416292 container died 4eaa1abfdbb5583564f346ae1179194d3b8432f6ab61a3796abad381319bc0b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:10:08.023 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:07 smithi136 podman[69672]: 2024-09-16 02:10:07.805789726 +0000 UTC m=+5.944241333 container cleanup 4eaa1abfdbb5583564f346ae1179194d3b8432f6ab61a3796abad381319bc0b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T02:10:08.023 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:07 smithi136 bash[69672]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10 2024-09-16T02:10:08.471 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:08 smithi136 podman[69687]: 2024-09-16 02:10:08.115185021 +0000 UTC m=+0.679239857 container remove 4eaa1abfdbb5583564f346ae1179194d3b8432f6ab61a3796abad381319bc0b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-16T02:10:08.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:08 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:08.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:08.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:08 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:08.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:08.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:08 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:08.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:08.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:08 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:08.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:08.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:08 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:08.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:08.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:08 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:08.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:08.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:08 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:08.058+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-09-16T02:09:41.081448+0000 front 2024-09-16T02:09:41.081511+0000 (oldest deadline 2024-09-16T02:10:06.381391+0000) 2024-09-16T02:10:08.760 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:08 smithi136 podman[69777]: 2024-09-16 02:10:08.373027702 +0000 UTC m=+0.015885780 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:10:08.760 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:08 smithi136 podman[69777]: 2024-09-16 02:10:08.474093241 +0000 UTC m=+0.116951351 container create 280206b0495edec9a58e4a82c93dbe18f8b5eee37e60f62819aa7df195ab18e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T02:10:08.760 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:08 smithi136 podman[69777]: 2024-09-16 02:10:08.684812207 +0000 UTC m=+0.327670275 container init 280206b0495edec9a58e4a82c93dbe18f8b5eee37e60f62819aa7df195ab18e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T02:10:08.761 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:08 smithi136 podman[69777]: 2024-09-16 02:10:08.688158392 +0000 UTC m=+0.331016461 container start 280206b0495edec9a58e4a82c93dbe18f8b5eee37e60f62819aa7df195ab18e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T02:10:08.761 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:08 smithi136 podman[69777]: 2024-09-16 02:10:08.759632273 +0000 UTC m=+0.402490358 container attach 280206b0495edec9a58e4a82c93dbe18f8b5eee37e60f62819aa7df195ab18e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:10:09.023 INFO:journalctl@ceph.osd.10.smithi136.stdout:Sep 16 02:10:08 smithi136 podman[69777]: 2024-09-16 02:10:08.875092371 +0000 UTC m=+0.517950449 container died 280206b0495edec9a58e4a82c93dbe18f8b5eee37e60f62819aa7df195ab18e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T02:10:09.422 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.10.service' 2024-09-16T02:10:09.461 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:09.462 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-16T02:10:09.462 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-16T02:10:09.462 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.11 2024-09-16T02:10:09.493 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:09.087+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:09.493 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:09.087+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:09.493 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:09.087+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:09.493 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:09.087+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:09.493 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:09.087+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:09.494 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:09.087+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:09.494 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:09.087+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-09-16T02:09:41.081448+0000 front 2024-09-16T02:09:41.081511+0000 (oldest deadline 2024-09-16T02:10:06.381391+0000) 2024-09-16T02:10:09.756 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 systemd[1]: Stopping Ceph osd.11 for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:10.045 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:09.753+0000 7fe94706a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:10.045 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:09.753+0000 7fe94706a640 -1 osd.11 84 *** Got signal Terminated *** 2024-09-16T02:10:10.045 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:09 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:09.753+0000 7fe94706a640 -1 osd.11 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:10:10.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:10.092+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:10.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:10.092+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:10.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:10.092+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:10.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:10.092+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:10.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:10.092+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:10.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:10.092+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:10.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:10 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:10.092+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-09-16T02:09:41.081448+0000 front 2024-09-16T02:09:41.081511+0000 (oldest deadline 2024-09-16T02:10:06.381391+0000) 2024-09-16T02:10:11.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:11 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:11.055+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:11.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:11 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:11.055+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:11.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:11 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:11.055+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:11.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:11 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:11.055+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:11.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:11 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:11.055+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:11.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:11 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:11.055+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:11.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:11 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:11.055+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-09-16T02:09:41.081448+0000 front 2024-09-16T02:09:41.081511+0000 (oldest deadline 2024-09-16T02:10:06.381391+0000) 2024-09-16T02:10:12.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:12 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:12.065+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:12.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:12 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:12.065+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:12.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:12 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:12.066+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:12.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:12 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:12.066+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:12.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:12 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:12.066+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:12.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:12 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:12.066+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:12.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:12 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:12.066+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-09-16T02:09:41.081448+0000 front 2024-09-16T02:09:41.081511+0000 (oldest deadline 2024-09-16T02:10:06.381391+0000) 2024-09-16T02:10:13.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:13 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:13.107+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:13.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:13 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:13.107+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:13.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:13 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:13.107+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:13.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:13 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:13.107+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:13.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:13 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:13.107+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:13.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:13 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:13.107+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:13.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:13 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:13.107+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-09-16T02:09:41.081448+0000 front 2024-09-16T02:09:41.081511+0000 (oldest deadline 2024-09-16T02:10:06.381391+0000) 2024-09-16T02:10:14.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:14.096+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-16T02:08:44.772510+0000 front 2024-09-16T02:08:44.772395+0000 (oldest deadline 2024-09-16T02:09:08.872321+0000) 2024-09-16T02:10:14.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:14.096+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-09-16T02:08:48.872785+0000 front 2024-09-16T02:08:48.872928+0000 (oldest deadline 2024-09-16T02:09:20.673500+0000) 2024-09-16T02:10:14.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:14.096+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-16T02:09:01.774266+0000 front 2024-09-16T02:09:01.774367+0000 (oldest deadline 2024-09-16T02:09:26.474343+0000) 2024-09-16T02:10:14.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:14.096+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-09-16T02:09:16.777518+0000 front 2024-09-16T02:09:16.777614+0000 (oldest deadline 2024-09-16T02:09:39.677515+0000) 2024-09-16T02:10:14.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:14.096+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-16T02:09:24.278768+0000 front 2024-09-16T02:09:24.278790+0000 (oldest deadline 2024-09-16T02:09:47.178927+0000) 2024-09-16T02:10:14.545 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:14.096+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-16T02:09:30.079975+0000 front 2024-09-16T02:09:30.079985+0000 (oldest deadline 2024-09-16T02:09:57.680330+0000) 2024-09-16T02:10:14.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:14.096+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-09-16T02:09:41.081448+0000 front 2024-09-16T02:09:41.081511+0000 (oldest deadline 2024-09-16T02:10:06.381391+0000) 2024-09-16T02:10:14.546 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:14 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11[60978]: 2024-09-16T02:10:14.096+0000 7fe94366f640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.172:6820 osd.8 since back 2024-09-16T02:09:48.082134+0000 front 2024-09-16T02:09:48.082167+0000 (oldest deadline 2024-09-16T02:10:13.982199+0000) 2024-09-16T02:10:15.364 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:15 smithi172 podman[69857]: 2024-09-16 02:10:15.085608545 +0000 UTC m=+5.424923635 container died b4b6ade518ddc40cf0dd64dacb306e51003dd4f6cf7c64769b0e718c4cdce5ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-16T02:10:15.764 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:15 smithi172 podman[69857]: 2024-09-16 02:10:15.470680701 +0000 UTC m=+5.809995778 container cleanup b4b6ade518ddc40cf0dd64dacb306e51003dd4f6cf7c64769b0e718c4cdce5ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:10:15.764 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:15 smithi172 bash[69857]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11 2024-09-16T02:10:16.045 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:15 smithi172 podman[69872]: 2024-09-16 02:10:15.763464583 +0000 UTC m=+0.674635577 container remove b4b6ade518ddc40cf0dd64dacb306e51003dd4f6cf7c64769b0e718c4cdce5ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:10:16.320 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:16 smithi172 podman[69958]: 2024-09-16 02:10:15.971995862 +0000 UTC m=+0.015826133 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:10:16.320 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:16 smithi172 podman[69958]: 2024-09-16 02:10:16.080914796 +0000 UTC m=+0.124745048 container create 812569ba99a36e1e3c970693c0673be05d687e0357ae71f95dcfc6145d776d41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T02:10:16.321 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:16 smithi172 podman[69958]: 2024-09-16 02:10:16.316628044 +0000 UTC m=+0.360458297 container init 812569ba99a36e1e3c970693c0673be05d687e0357ae71f95dcfc6145d776d41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T02:10:16.321 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:16 smithi172 podman[69958]: 2024-09-16 02:10:16.319790782 +0000 UTC m=+0.363621035 container start 812569ba99a36e1e3c970693c0673be05d687e0357ae71f95dcfc6145d776d41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T02:10:16.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:16 smithi172 podman[69958]: 2024-09-16 02:10:16.416800636 +0000 UTC m=+0.460630897 container attach 812569ba99a36e1e3c970693c0673be05d687e0357ae71f95dcfc6145d776d41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T02:10:16.795 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 16 02:10:16 smithi172 podman[69958]: 2024-09-16 02:10:16.505783778 +0000 UTC m=+0.549614045 container died 812569ba99a36e1e3c970693c0673be05d687e0357ae71f95dcfc6145d776d41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T02:10:17.104 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@osd.11.service' 2024-09-16T02:10:17.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:17.184 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-16T02:10:17.184 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-16T02:10:17.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.a 2024-09-16T02:10:17.504 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:10:17 smithi103 systemd[1]: Stopping Ceph mds.a for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:18.005 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:10:17 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-a[71697]: 2024-09-16T02:10:17.615+0000 7f1aff400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:18.005 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:10:17 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-a[71697]: 2024-09-16T02:10:17.615+0000 7f1aff400640 -1 mds.a *** got signal Terminated *** 2024-09-16T02:10:18.961 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:10:18 smithi103 podman[89647]: 2024-09-16 02:10:18.802814668 +0000 UTC m=+1.323557364 container died ddd596b8db3cf4df00af744da834beb82d7d2a9ff54ca2bef02525617bdc813e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-16T02:10:19.564 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:10:19 smithi103 podman[89647]: 2024-09-16 02:10:19.286780308 +0000 UTC m=+1.807523004 container cleanup ddd596b8db3cf4df00af744da834beb82d7d2a9ff54ca2bef02525617bdc813e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:10:19.565 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 16 02:10:19 smithi103 bash[89647]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-a 2024-09-16T02:10:19.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.a.service' 2024-09-16T02:10:19.834 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:19.835 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-16T02:10:19.835 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-16T02:10:19.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.d 2024-09-16T02:10:20.253 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:10:19 smithi103 systemd[1]: Stopping Ceph mds.d for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:20.505 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:10:20 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-d[70519]: 2024-09-16T02:10:20.252+0000 7f2600a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:20.505 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:10:20 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-d[70519]: 2024-09-16T02:10:20.252+0000 7f2600a00640 -1 mds.d *** got signal Terminated *** 2024-09-16T02:10:23.810 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:10:23 smithi103 podman[89781]: 2024-09-16 02:10:23.514389837 +0000 UTC m=+3.392272901 container died 3a94b3dabb4f410d79d44d15334c497c2b580dc2426c436b3b11eae022f4e428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-16T02:10:24.226 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:10:23 smithi103 podman[89781]: 2024-09-16 02:10:23.990805953 +0000 UTC m=+3.868689015 container cleanup 3a94b3dabb4f410d79d44d15334c497c2b580dc2426c436b3b11eae022f4e428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:10:24.226 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 16 02:10:23 smithi103 bash[89781]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-d 2024-09-16T02:10:24.390 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.d.service' 2024-09-16T02:10:24.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:24.460 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-16T02:10:24.461 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-16T02:10:24.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.g 2024-09-16T02:10:24.866 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:10:24 smithi103 systemd[1]: Stopping Ceph mds.g for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:25.254 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:10:24 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-g[71321]: 2024-09-16T02:10:24.864+0000 7f9cbec00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:25.255 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:10:24 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-g[71321]: 2024-09-16T02:10:24.864+0000 7f9cbec00640 -1 mds.g *** got signal Terminated *** 2024-09-16T02:10:27.746 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:10:27 smithi103 podman[89900]: 2024-09-16 02:10:27.454989821 +0000 UTC m=+2.702428039 container died e4778ffffc52ec1cfe11072b1d98a23ab8b58809e657797b3f88b3b60005882a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS) 2024-09-16T02:10:28.004 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:10:27 smithi103 podman[89900]: 2024-09-16 02:10:27.953274418 +0000 UTC m=+3.200712636 container cleanup e4778ffffc52ec1cfe11072b1d98a23ab8b58809e657797b3f88b3b60005882a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T02:10:28.005 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 16 02:10:27 smithi103 bash[89900]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-g 2024-09-16T02:10:28.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.g.service' 2024-09-16T02:10:28.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:28.449 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-16T02:10:28.450 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-16T02:10:28.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.j 2024-09-16T02:10:28.754 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:10:28 smithi103 systemd[1]: Stopping Ceph mds.j for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:29.254 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:10:28 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-j[70107]: 2024-09-16T02:10:28.842+0000 7f44a3e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:29.255 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:10:28 smithi103 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-j[70107]: 2024-09-16T02:10:28.842+0000 7f44a3e00640 -1 mds.j *** got signal Terminated *** 2024-09-16T02:10:31.893 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:10:31 smithi103 podman[90018]: 2024-09-16 02:10:31.633250026 +0000 UTC m=+2.920646617 container died c975b03dbfa12486c55935749c0043750ea8484da7ddf3d496a105ac0f3af60a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T02:10:32.175 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:10:32 smithi103 podman[90018]: 2024-09-16 02:10:32.007055202 +0000 UTC m=+3.294451792 container cleanup c975b03dbfa12486c55935749c0043750ea8484da7ddf3d496a105ac0f3af60a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T02:10:32.175 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:10:32 smithi103 bash[90018]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-j 2024-09-16T02:10:32.427 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 16 02:10:32 smithi103 podman[90030]: 2024-09-16 02:10:32.174578546 +0000 UTC m=+0.533902728 container remove c975b03dbfa12486c55935749c0043750ea8484da7ddf3d496a105ac0f3af60a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-j, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T02:10:32.436 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.j.service' 2024-09-16T02:10:32.505 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:32.505 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-16T02:10:32.505 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-16T02:10:32.506 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.b 2024-09-16T02:10:32.955 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:10:32 smithi136 systemd[1]: Stopping Ceph mds.b for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:33.273 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:10:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-b[66066]: 2024-09-16T02:10:32.952+0000 7f959a818640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:33.273 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:10:32 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-b[66066]: 2024-09-16T02:10:32.952+0000 7f959a818640 -1 mds.b *** got signal Terminated *** 2024-09-16T02:10:37.238 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:10:36 smithi136 podman[69893]: 2024-09-16 02:10:36.95961112 +0000 UTC m=+4.253054445 container died a47b2ae2e9126a1eb89bb840b313733a8ce48747574c6e93270e5b69a8dde56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T02:10:37.523 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:10:37 smithi136 podman[69893]: 2024-09-16 02:10:37.327883851 +0000 UTC m=+4.621327175 container cleanup a47b2ae2e9126a1eb89bb840b313733a8ce48747574c6e93270e5b69a8dde56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-b, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:10:37.523 INFO:journalctl@ceph.mds.b.smithi136.stdout:Sep 16 02:10:37 smithi136 bash[69893]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-b 2024-09-16T02:10:37.711 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.b.service' 2024-09-16T02:10:37.747 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:37.747 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-16T02:10:37.747 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-16T02:10:37.747 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.e 2024-09-16T02:10:38.023 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:10:37 smithi136 systemd[1]: Stopping Ceph mds.e for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:38.523 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:10:38 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e[65457]: 2024-09-16T02:10:38.072+0000 7f8aa66e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:38.523 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:10:38 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e[65457]: 2024-09-16T02:10:38.072+0000 7f8aa66e3640 -1 mds.e *** got signal Terminated *** 2024-09-16T02:10:41.481 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:10:41 smithi136 podman[70011]: 2024-09-16 02:10:41.153836313 +0000 UTC m=+3.193226799 container died f79535200b0ae785a43503e5ffdb094c38bcfdd7ffe8ecf87c07e2fe722f4f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T02:10:41.749 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:10:41 smithi136 podman[70011]: 2024-09-16 02:10:41.606035388 +0000 UTC m=+3.645425873 container cleanup f79535200b0ae785a43503e5ffdb094c38bcfdd7ffe8ecf87c07e2fe722f4f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:10:41.750 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:10:41 smithi136 bash[70011]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e 2024-09-16T02:10:42.023 INFO:journalctl@ceph.mds.e.smithi136.stdout:Sep 16 02:10:41 smithi136 podman[70023]: 2024-09-16 02:10:41.899049396 +0000 UTC m=+0.742644674 container remove f79535200b0ae785a43503e5ffdb094c38bcfdd7ffe8ecf87c07e2fe722f4f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:10:42.043 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.e.service' 2024-09-16T02:10:42.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:42.078 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-16T02:10:42.078 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-16T02:10:42.079 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.h 2024-09-16T02:10:42.520 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:10:42 smithi136 systemd[1]: Stopping Ceph mds.h for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:42.773 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:10:42 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h[65765]: 2024-09-16T02:10:42.517+0000 7f9229aa1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:42.773 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:10:42 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h[65765]: 2024-09-16T02:10:42.517+0000 7f9229aa1640 -1 mds.h *** got signal Terminated *** 2024-09-16T02:10:45.519 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:10:45 smithi136 podman[70140]: 2024-09-16 02:10:45.149583192 +0000 UTC m=+2.851248088 container died 5bd2a4f82897dc0acae37cdafe5b797b13ec825f73b14a9ac9410eab76c695ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T02:10:45.773 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:10:45 smithi136 podman[70140]: 2024-09-16 02:10:45.601222684 +0000 UTC m=+3.302887575 container cleanup 5bd2a4f82897dc0acae37cdafe5b797b13ec825f73b14a9ac9410eab76c695ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T02:10:45.773 INFO:journalctl@ceph.mds.h.smithi136.stdout:Sep 16 02:10:45 smithi136 bash[70140]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-h 2024-09-16T02:10:46.025 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.h.service' 2024-09-16T02:10:46.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:46.059 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-16T02:10:46.059 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-16T02:10:46.059 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.k 2024-09-16T02:10:46.523 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:10:46 smithi136 systemd[1]: Stopping Ceph mds.k for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:46.523 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:10:46 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-k[65148]: 2024-09-16T02:10:46.379+0000 7f811f725640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:46.523 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:10:46 smithi136 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-k[65148]: 2024-09-16T02:10:46.379+0000 7f811f725640 -1 mds.k *** got signal Terminated *** 2024-09-16T02:10:48.197 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:10:47 smithi136 podman[70261]: 2024-09-16 02:10:47.861685179 +0000 UTC m=+1.584697163 container died 3c88d7cf0f4b03210a3f3abbdca8df55a6db9ade1f08ff5913706e77ebd1d6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T02:10:48.456 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:10:48 smithi136 podman[70261]: 2024-09-16 02:10:48.279307091 +0000 UTC m=+2.002319073 container cleanup 3c88d7cf0f4b03210a3f3abbdca8df55a6db9ade1f08ff5913706e77ebd1d6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-k, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T02:10:48.456 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:10:48 smithi136 bash[70261]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-k 2024-09-16T02:10:48.697 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.k.service' 2024-09-16T02:10:48.719 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:10:48 smithi136 podman[70273]: 2024-09-16 02:10:48.45522524 +0000 UTC m=+0.586402805 container remove 3c88d7cf0f4b03210a3f3abbdca8df55a6db9ade1f08ff5913706e77ebd1d6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-16T02:10:48.719 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:10:48 smithi136 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.k.service: Deactivated successfully. 2024-09-16T02:10:48.719 INFO:journalctl@ceph.mds.k.smithi136.stdout:Sep 16 02:10:48 smithi136 systemd[1]: Stopped Ceph mds.k for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:10:48.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:48.730 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-16T02:10:48.730 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-16T02:10:48.730 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.c 2024-09-16T02:10:49.028 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:10:48 smithi172 systemd[1]: Stopping Ceph mds.c for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:49.295 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:10:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c[65852]: 2024-09-16T02:10:49.025+0000 7fd103ad5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:49.295 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:10:49 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c[65852]: 2024-09-16T02:10:49.025+0000 7fd103ad5640 -1 mds.c *** got signal Terminated *** 2024-09-16T02:10:53.295 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:10:53 smithi172 podman[70059]: 2024-09-16 02:10:53.033419047 +0000 UTC m=+4.105862039 container died 1a8f64a9b35339a0060835a6d7abc9071369d59b0deeccec716597ec678a001b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-16T02:10:53.670 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:10:53 smithi172 podman[70059]: 2024-09-16 02:10:53.401623044 +0000 UTC m=+4.474066035 container cleanup 1a8f64a9b35339a0060835a6d7abc9071369d59b0deeccec716597ec678a001b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T02:10:53.670 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 16 02:10:53 smithi172 bash[70059]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-c 2024-09-16T02:10:53.840 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.c.service' 2024-09-16T02:10:53.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:53.877 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-16T02:10:53.877 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-16T02:10:53.877 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.f 2024-09-16T02:10:54.198 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:10:53 smithi172 systemd[1]: Stopping Ceph mds.f for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:54.545 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:10:54 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f[65536]: 2024-09-16T02:10:54.196+0000 7f439c7f0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:54.545 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:10:54 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f[65536]: 2024-09-16T02:10:54.196+0000 7f439c7f0640 -1 mds.f *** got signal Terminated *** 2024-09-16T02:10:57.449 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:10:57 smithi172 podman[70191]: 2024-09-16 02:10:57.151854135 +0000 UTC m=+3.061311350 container died 54d1f2756558e70ca41c922fe8a9a1ce238c88f4e23b1aa72675c5c6fec6944c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2) 2024-09-16T02:10:57.782 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:10:57 smithi172 podman[70191]: 2024-09-16 02:10:57.521971184 +0000 UTC m=+3.431428396 container cleanup 54d1f2756558e70ca41c922fe8a9a1ce238c88f4e23b1aa72675c5c6fec6944c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T02:10:57.782 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 16 02:10:57 smithi172 bash[70191]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-f 2024-09-16T02:10:57.947 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.f.service' 2024-09-16T02:10:57.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:10:57.980 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-16T02:10:57.980 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-16T02:10:57.980 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.i 2024-09-16T02:10:58.295 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:10:58 smithi172 systemd[1]: Stopping Ceph mds.i for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:10:58.795 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:10:58 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i[66149]: 2024-09-16T02:10:58.300+0000 7f82acdab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:10:58.795 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:10:58 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i[66149]: 2024-09-16T02:10:58.300+0000 7f82acdab640 -1 mds.i *** got signal Terminated *** 2024-09-16T02:11:01.545 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:11:01 smithi172 podman[70312]: 2024-09-16 02:11:01.156485478 +0000 UTC m=+2.956390603 container died bb31bc2221b4b62047833ffc76dd151aa14eafe87630a44607e7db3743d0cf00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T02:11:01.885 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:11:01 smithi172 podman[70312]: 2024-09-16 02:11:01.700378229 +0000 UTC m=+3.500283346 container cleanup bb31bc2221b4b62047833ffc76dd151aa14eafe87630a44607e7db3743d0cf00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T02:11:01.886 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:11:01 smithi172 bash[70312]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i 2024-09-16T02:11:02.125 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.i.service' 2024-09-16T02:11:02.147 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:11:01 smithi172 podman[70324]: 2024-09-16 02:11:01.884871593 +0000 UTC m=+0.724778812 container remove bb31bc2221b4b62047833ffc76dd151aa14eafe87630a44607e7db3743d0cf00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T02:11:02.147 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:11:02 smithi172 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-16T02:11:02.148 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:11:02 smithi172 systemd[1]: Stopped Ceph mds.i for a20dd7a4-73ce-11ef-bceb-c7b262605968. 2024-09-16T02:11:02.148 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 16 02:11:02 smithi172 systemd[1]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.i.service: Consumed 1.326s CPU time. 2024-09-16T02:11:02.159 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:11:02.159 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-16T02:11:02.159 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-16T02:11:02.159 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.l 2024-09-16T02:11:02.472 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:11:02 smithi172 systemd[1]: Stopping Ceph mds.l for a20dd7a4-73ce-11ef-bceb-c7b262605968... 2024-09-16T02:11:02.795 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:11:02 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-l[65232]: 2024-09-16T02:11:02.470+0000 7fc482bd7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:11:02.795 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:11:02 smithi172 ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-l[65232]: 2024-09-16T02:11:02.470+0000 7fc482bd7640 -1 mds.l *** got signal Terminated *** 2024-09-16T02:11:05.795 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:11:05 smithi172 podman[70432]: 2024-09-16 02:11:05.495060911 +0000 UTC m=+3.124486421 container died 1f7e1640479fe8671d97b6a3e22480d77e17c1028c9a5f40fc702661072420da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T02:11:06.139 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:11:05 smithi172 podman[70432]: 2024-09-16 02:11:05.879022089 +0000 UTC m=+3.508447600 container cleanup 1f7e1640479fe8671d97b6a3e22480d77e17c1028c9a5f40fc702661072420da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T02:11:06.139 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 16 02:11:05 smithi172 bash[70432]: ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968-mds-l 2024-09-16T02:11:06.258 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a20dd7a4-73ce-11ef-bceb-c7b262605968@mds.l.service' 2024-09-16T02:11:06.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:11:06.290 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-16T02:11:06.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T02:11:06.566 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T02:11:10.423 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T02:11:10.626 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T02:11:13.071 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T02:11:13.273 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T02:11:15.816 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:11:15.879 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:11:15.913 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:11:15.939 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T02:11:15.941 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496/remote/smithi103/crash 2024-09-16T02:11:15.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/crash -- . 2024-09-16T02:11:16.016 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T02:11:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-09-16T02:11:16.020 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496/remote/smithi136/crash 2024-09-16T02:11:16.021 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/crash -- . 2024-09-16T02:11:16.063 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T02:11:16.063 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-09-16T02:11:16.065 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496/remote/smithi172/crash 2024-09-16T02:11:16.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/crash -- . 2024-09-16T02:11:16.106 INFO:teuthology.orchestra.run.smithi172.stderr:tar: /var/lib/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T02:11:16.106 INFO:teuthology.orchestra.run.smithi172.stderr:tar: Error is not recoverable: exiting now 2024-09-16T02:11:16.107 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T02:11:16.107 DEBUG:teuthology.orchestra.run.smithi103:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-16T02:11:16.188 INFO:tasks.cephadm:Compressing logs... 2024-09-16T02:11:16.188 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T02:11:16.192 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T02:11:16.194 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T02:11:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T02:11:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:: No such file or directory 2024-09-16T02:11:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T02:11:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-16T02:11:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T02:11:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T02:11:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-16T02:11:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T02:11:16.223 INFO:teuthology.orchestra.run.smithi172.stderr: 82.6% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T02:11:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T02:11:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.log 2024-09-16T02:11:16.223 INFO:teuthology.orchestra.run.smithi172.stderr: 90.5% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T02:11:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-16T02:11:16.224 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-16T02:11:16.225 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.l.log: 86.3% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T02:11:16.225 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-16T02:11:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-16T02:11:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:: No such file or directory 2024-09-16T02:11:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T02:11:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T02:11:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T02:11:16.228 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.8.log: 94.8% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-16T02:11:16.228 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-16T02:11:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-16T02:11:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-16T02:11:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T02:11:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T02:11:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T02:11:16.232 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.audit.log: 94.6% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-16T02:11:16.232 INFO:teuthology.orchestra.run.smithi136.stderr: 90.5% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T02:11:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T02:11:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose 82.7% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T02:11:16.233 INFO:teuthology.orchestra.run.smithi136.stderr: -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.log 2024-09-16T02:11:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-16T02:11:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mgr.y.log: /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.log: 86.2% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T02:11:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T02:11:16.235 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mgr.z.log: 92.2% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-16T02:11:16.235 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-16T02:11:16.235 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T02:11:16.235 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T02:11:16.236 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.h.log: 92.1% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T02:11:16.236 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-16T02:11:16.237 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T02:11:16.242 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T02:11:16.242 INFO:teuthology.orchestra.run.smithi103.stderr:: No such file or directory 2024-09-16T02:11:16.243 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-16T02:11:16.244 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T02:11:16.245 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T02:11:16.246 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-16T02:11:16.247 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T02:11:16.248 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-16T02:11:16.249 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T02:11:16.250 INFO:teuthology.orchestra.run.smithi103.stderr: 90.3% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T02:11:16.250 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-16T02:11:16.250 INFO:teuthology.orchestra.run.smithi103.stderr: 82.7% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T02:11:16.250 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T02:11:16.252 INFO:teuthology.orchestra.run.smithi103.stderr: 94.6% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-16T02:11:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.log 2024-09-16T02:11:16.253 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T02:11:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-16T02:11:16.255 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T02:11:16.255 INFO:teuthology.orchestra.run.smithi103.stderr: 94.6% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-16T02:11:16.257 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.d.log: 86.3% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T02:11:16.257 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-16T02:11:16.261 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mgr.x.log: /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.4% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-16T02:11:16.261 INFO:teuthology.orchestra.run.smithi103.stderr: 91.0% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T02:11:16.261 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T02:11:16.262 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T02:11:16.263 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.3.log: 94.6% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-16T02:11:16.263 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T02:11:16.268 INFO:teuthology.orchestra.run.smithi172.stderr: 92.1% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-16T02:11:16.269 INFO:teuthology.orchestra.run.smithi172.stderr: 94.0% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-16T02:11:16.274 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/cephadm.log: 93.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T02:11:16.278 INFO:teuthology.orchestra.run.smithi136.stderr: 92.4% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-16T02:11:16.287 INFO:teuthology.orchestra.run.smithi136.stderr: 93.8% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-16T02:11:16.470 INFO:teuthology.orchestra.run.smithi103.stderr: 90.0% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T02:11:16.512 INFO:teuthology.orchestra.run.smithi172.stderr: 91.7% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T02:11:16.527 INFO:teuthology.orchestra.run.smithi136.stderr: 91.7% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T02:11:16.658 INFO:teuthology.orchestra.run.smithi172.stderr: 95.8% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-16T02:11:16.710 INFO:teuthology.orchestra.run.smithi172.stderr: 93.5% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T02:11:16.750 INFO:teuthology.orchestra.run.smithi172.stderr: 93.7% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-16T02:11:16.760 INFO:teuthology.orchestra.run.smithi103.stderr: 93.0% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-16T02:11:16.776 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T02:11:16.780 INFO:teuthology.orchestra.run.smithi103.stderr: 90.6% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T02:11:16.814 INFO:teuthology.orchestra.run.smithi103.stderr: 93.8% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T02:11:16.821 INFO:teuthology.orchestra.run.smithi103.stderr: 93.1% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T02:11:16.832 INFO:teuthology.orchestra.run.smithi136.stderr: 93.3% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T02:11:16.857 INFO:teuthology.orchestra.run.smithi136.stderr: 93.7% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-16T02:11:16.870 INFO:teuthology.orchestra.run.smithi136.stderr: 93.5% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T02:11:16.877 INFO:teuthology.orchestra.run.smithi103.stderr: 93.0% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T02:11:16.880 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-16T02:11:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m0.659s 2024-09-16T02:11:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:user 0m2.806s 2024-09-16T02:11:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m0.329s 2024-09-16T02:11:16.906 INFO:teuthology.orchestra.run.smithi172.stderr: 93.7% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-16T02:11:18.347 INFO:teuthology.orchestra.run.smithi172.stderr: 96.3% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T02:11:18.349 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-09-16T02:11:18.349 INFO:teuthology.orchestra.run.smithi172.stderr:real 0m2.140s 2024-09-16T02:11:18.349 INFO:teuthology.orchestra.run.smithi172.stderr:user 0m4.486s 2024-09-16T02:11:18.349 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m0.189s 2024-09-16T02:11:19.698 INFO:teuthology.orchestra.run.smithi136.stderr: 93.5% -- replaced with /var/log/ceph/a20dd7a4-73ce-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-16T02:11:19.700 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-09-16T02:11:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m3.491s 2024-09-16T02:11:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m6.109s 2024-09-16T02:11:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m0.242s 2024-09-16T02:11:19.701 INFO:tasks.cephadm:Archiving logs... 2024-09-16T02:11:19.701 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496/remote/smithi103/log 2024-09-16T02:11:19.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T02:11:20.151 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496/remote/smithi136/log 2024-09-16T02:11:20.152 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T02:11:21.099 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496/remote/smithi172/log 2024-09-16T02:11:21.100 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T02:11:21.640 INFO:tasks.cephadm:Removing cluster... 2024-09-16T02:11:21.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 --force 2024-09-16T02:11:21.912 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T02:11:22.342 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 --force 2024-09-16T02:11:22.544 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T02:11:22.813 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a20dd7a4-73ce-11ef-bceb-c7b262605968 --force 2024-09-16T02:11:23.004 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: a20dd7a4-73ce-11ef-bceb-c7b262605968 2024-09-16T02:11:23.265 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T02:11:23.265 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T02:11:23.302 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T02:11:23.319 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T02:11:23.336 INFO:tasks.cephadm:Teardown complete 2024-09-16T02:11:23.336 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T02:11:23.347 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T02:11:23.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T02:11:23.352 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T02:11:23.362 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T02:11:23.478 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T02:11:23.488 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T02:11:23.489 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T02:11:23.494 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T02:11:23.496 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T02:11:23.508 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-09-16T02:11:23.510 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-09-16T02:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T02:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-09-16T02:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 -452us[ -422us] +/- 47ms 2024-09-16T02:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 +455us[ +484us] +/- 58ms 2024-09-16T02:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +146us[ +176us] +/- 71ms 2024-09-16T02:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T02:11:23.514 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T02:11:23.514 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-09-16T02:11:23.514 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 -452us[ -451us] +/- 47ms 2024-09-16T02:11:23.515 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 +457us[ +457us] +/- 58ms 2024-09-16T02:11:23.515 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +156us[ +156us] +/- 71ms 2024-09-16T02:11:23.515 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T02:11:23.523 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-16T02:11:23.530 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T02:11:23.530 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-16T02:11:23.530 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 -430us[ -458us] +/- 47ms 2024-09-16T02:11:23.530 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 +504us[ +504us] +/- 58ms 2024-09-16T02:11:23.530 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +169us[ +141us] +/- 71ms 2024-09-16T02:11:23.531 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T02:11:23.531 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T02:11:23.541 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T02:11:23.542 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T02:11:23.551 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T02:11:23.697 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T02:11:23.765 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T02:11:23.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T02:11:23.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:11:23.895 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T02:11:23.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:11:23.923 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T02:11:23.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:11:23.951 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T02:11:23.961 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T02:11:23.976 INFO:teuthology.task.internal:Duration was 1393.886244 seconds 2024-09-16T02:11:23.976 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T02:11:23.987 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T02:11:23.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T02:11:23.992 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T02:11:23.994 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T02:11:24.030 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T02:11:24.034 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T02:11:24.075 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T02:11:24.559 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T02:11:24.559 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-09-16T02:11:24.559 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T02:11:24.608 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T02:11:24.608 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T02:11:24.632 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-09-16T02:11:24.632 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T02:11:24.658 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T02:11:24.659 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T02:11:24.663 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T02:11:24.675 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T02:11:24.727 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T02:11:24.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T02:11:24.731 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T02:11:24.742 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T02:11:25.353 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T02:11:25.365 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T02:11:25.365 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T02:11:25.421 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T02:11:25.448 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T02:11:25.475 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T02:11:25.486 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:11:25.490 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:11:25.493 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:11:25.515 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-09-16T02:11:25.539 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-09-16T02:11:25.556 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-09-16T02:11:25.586 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:11:25.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:11:25.619 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:11:25.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:11:25.634 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:11:25.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:11:25.649 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T02:11:25.661 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T02:11:25.661 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496/remote/smithi103 2024-09-16T02:11:25.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T02:11:25.740 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496/remote/smithi136 2024-09-16T02:11:25.740 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T02:11:25.776 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906496/remote/smithi172 2024-09-16T02:11:25.776 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T02:11:25.812 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T02:11:25.812 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T02:11:25.817 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T02:11:25.819 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T02:11:25.862 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T02:11:25.875 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T02:11:25.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T02:11:25.885 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T02:11:25.886 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T02:11:25.893 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T02:11:25.895 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T02:11:25.908 INFO:teuthology.orchestra.run.smithi136.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 02:11 /home/ubuntu/cephtest 2024-09-16T02:11:25.917 INFO:teuthology.orchestra.run.smithi172.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 02:11 /home/ubuntu/cephtest 2024-09-16T02:11:25.924 INFO:teuthology.orchestra.run.smithi103.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 02:11 /home/ubuntu/cephtest 2024-09-16T02:11:25.926 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T02:11:25.938 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T02:11:25.967 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/direct_io}} duration: 1393.886244058609 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T02:11:25.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T02:11:26.043 INFO:teuthology.run:pass